the International Journal Volume 15

on Marine Navigation Number 4

http://www.transnav.eu and Safety of Sea Transportation December 2021

DOI: 10.12716/1001.15.04.03

Assessment of the Concept of the New Methodology

for the Evaluation of Ship Planned Maintenance System
T. Stanivuk, L. Stazić, F. Vidović & K. Bratić
University of Split, Split, Croatia

ABSTRACT: The paper presents an attempt to develop better and cheaper methodology for evaluation of the
quality of the ships’ computerized Planned Maintenance System databases. A concept for the better
methodology has been developed following many times verified rule that faster and cheaper is better. The
concept of the methodology has been checked in operation, tested on four different databases. Results of the
evaluation performed by the concept are compared with an expert evaluation using old, verified methodology,
performed on same databases. Comparison valuated the concept, bringing the verdict if it is functioning or not,
showing that the old and verified rule is not functioning every time.

1 INTRODUCTION Another drawback of the evaluation methodology is

that evaluation takes several hours per database.
The creation of a methodology for an evaluation is Therefore, the evaluation of the quality of the PMS
process which takes several steps from defining the database become costly operation and shipping
problem, developing the solution and finally testing companies quite often accept PMS database without
the operability of the solution [10]. Fully operational proper and efficient quality control. Drawbacks in the
evaluation methodology for the quality of the methodology and the need for constant improvement
Computerized PMS (Planned Maintenance Systems) [8] were a driving force for the development of the
for ships has been developed in 2017 [12]. After new methodology which will be without listed flaws.
development, the methodology has been tested and
released for free use. Since then, quite a large number
of databases has been evaluated using the 1.1 Non-disclosure condition
methodology, either for scientific or for economic
purposes. Shipping companies allowed access to their data
strictly under no disclosure condition. All
Several unwanted characteristics or flaws were identification details (either the ship or the company)
identified during the use of the methodology. The are removed from the article.
process of the evaluation requires an expert with
advanced knowledge of the marine engineering
(senior engineering rank), knowledge of the Company
structure and policies (usually from the company or
very well versed about the Company) and advanced
knowledge of the used computerized PMS system.

739
2 THE METHODOLOGY CONCEPT “Database on average is in order, database does
not require immediate action. There are some areas
Development of the solution was the next step after where action is required:
defining the problem [10]. A concept of the new − some components are without spare parts,
evaluation methodology has been created, following − maintenance plan on some components should be
the idea that the simpler is the better [1], with improved,
intention to be simple enough that an ordinary skilled − some components do not have defined equipment
person can evaluate any PMS database without details”.
preparation or special knowledge. Another concept Obtained results were very promising although
design condition was based on the principle that conclusion was not formed because “small sample
faster is better [2], the evaluation time of the database should not lead to great conclusions” [13].
quality is shortened to less than half an hour. Another
condition introduced for the concept was that it
should be operable on any type of computerized PMS,
obtaining same results. 4 THE SECOND STAGE TEST
Several researches pointed that most common
deficiency in the computerized databases was missing Further testing has been performed, hoping that
information [3, 6]. In line with that idea, the concept results will look similar to the results of the initial
for the new methodology is trying to establish link database, which will verify functioning of the concept.
between the quality and the quantity of the data in The evaluation of three ship databases was divided
ships’ computerized PMS databases. To minimize the into two separate evaluations [9].
time needed for the evaluation, only a randomly The first evaluation was performed using the
chosen sample of the database should be analysed, not concept by an ordinary skilled evaluator. Second
the whole database [4, 5]. This will ensure that the quality evaluation of computerized PMS databases
time for the evaluation is short and cheap. In this case, was performed on same databases by an expert using
several pumps, randomly chosen should act as a old methodology. The evaluation was performed on
sample for the testing. the premises of the shipping company. Analysed
The basic idea of the new concept is very simple, it vessels are not sisterships, data collected using the
is basic. The idea stems from already stated fact that concept is presented in Tables 1, 2, 3.
the most common deficiency in computerized Table 1. Results of information counting of the database A
databases is a missing of information [3, 6]. Therefore, _______________________________________________
the concept is derived from the idea that the larger Equipment name Equip. Spares Spare Purch. Work
details number details data data
_______________________________________________
amount of information in a database means that the
database has fewer flaws. If this is true, the evaluation SW cooling pump 5 47 6 2 3
of the quality of the ship’s computerized Planned FO Transfer pump 4 40 5 2 2
Maintenance database could be reduced to a mere Firefighting pump 6 49 6 2 5
Bilge/ball. pump 6 43 6 2 3
counting of the data in the database. This procedure Cargo pump 6 117 6 2 2
does not require knowledge of systems and Em’cy FF pump 6 49 6 2 5
mechanical engineering and can be performed by Bilge piston pump 5 40 5 2 2
anyone. It would create significant savings in the _______________________________________________
AVERAGE 5.4 55 5.7 2 3.1
_______________________________________________
process of the control and the development of the
database and reduce the total cost of the system.
Table 2. Results of information counting of the database B
_______________________________________________
Testing the operability of the solution (concept) Equipment name Equip. Spares Spare Purch. Work
was performed in two stages, each stage had separate details number details data data
research objectives. The objective of the first stage [11] _______________________________________________
SW cooling pump 6 42 5 2 4
was to determine the functionality of the concept.
FO Transfer pump 6 37 6 2 2
Second stage of the testing had to determine if results Firefighting pump 6 44 6 2 3
obtained by concept are corresponding to the quality Bilge/ball. Pump 4 46 5 2 4
of the DB established by an expert. Cargo pump 5 92 5 2 3
Em’cy FF pump 4 39 4 2 3
Bilge piston pump 5 46 4 2 3
_______________________________________________
AVERAGE 5.1 49.4 5 2 3.1
_______________________________________________
3 THE FIRST STAGE TEST
Table 3. Results of information counting of the database C
_______________________________________________
In order to determine the functionality of the concept,
a randomly chosen evaluator, with average skills, Equipment name Equip. Spares Spare Purch. Work
details number details data data
_______________________________________________
performed evaluation of a database. Database
evaluation was performed on the Company premises, SW cooling pump 5 51 4 2 5
by a student of Marine Engineering. FO Transfer pump 6 44 5 2 3
Firefighting pump 5 36 6 2 4
Overall evaluation of the database [11], based on Bilge/ball. pump 5 38 6 2 3
the concept, was that DB quality is good, with note Cargo pump 6 108 6 2 5
that improvement should be performed with spare Em’cy FF pump 5 42 5 2 4
parts and equipment details and maintenance plan. Bilge piston pump 6 45 4 2 4
_______________________________________________
AVERAGE 5.4 52 5.1 2 4
_______________________________________________
Results of the concept testing were compared with
the expert evaluation, performed in 2017 [13]:

740
 The evaluator (with ordinary skills) analysed together with the testing plan. Spare parts are
obtained results and formed overall evaluation about linked to all components, together with purchase
PMS databases. The overall evaluation of databases is data and details.
summarized: − Two areas require immediate attention; PMS self-
− DB 1 (database 1) has all chosen components; all of improvement program has to be established as
components have linked equipment details. Spare soon as possible in order to report and supervise
parts are linked to all components, maintenance DB and its functioning and alarm testing program
plan is well designed and tuned. Spare parts needs to be inserted in the DB to enable
details are present as well as extensive purchase supervision of this segment. Another three items
data. There are no areas for improvement found are with intermediate priority, improvement is
during this evaluation. also needed but that does not have to be performed
− DB 2 has all chosen components; all of components as soon as possible. There is a multitude of
have linked equipment details. Spare parts are examples in databases where same job is scheduled
linked to all components, maintenance plan is well twice, for example an overhaul is required
designed and tuned. Spare parts details are present (scheduled) due to manufacturer recommendation,
as well as extensive purchase data. There are no at the same time there is overhaul scheduled due
areas for improvement found during this to Class Survey.
evaluation. − Databases also have examples where same job is
− DB 3 has all chosen components; all of components inserted twice, for work order for example electric
have linked equipment details. Spare parts are motor overhaul is linked to pump and to its motor
linked to all components, maintenance plan is well as well. Also, it is noted that work frequency is not
designed and tuned. Spare parts details are present synchronized, i.e., work orders should be grouped
as well as extensive purchase data. There are no to avoid unnecessary paperwork.
areas for improvement found during this
evaluation.
An expert evaluated same databases using the old 5 DISCUSSION
methodology and the questionnaire [12]. Results of
those evaluations are almost the same for all three
The evaluator with ordinary skills using new concept
databases and are presented in the Table 4 (Results are
analysed computerized PMS databases and the
applicable for all three databases). Only major
duration of that evaluation was 30 minutes per
deficiencies are collected in the table, i.e. areas marked
database or 90. Duration of the evaluation performed
with low grades. There are also some other areas
by expert was much longer, almost four hours per
where noted deficiencies are minor and no
database. Also, the expert's report and discoveries are
improvement was recommended by the expert.
much more extensive and detailed as expected [7],
Table 4. Old methodology analysis results for all databases
_______________________________________________ and it contains a whole series of observations that are
Question Priority Grade not present in the opinion of an ordinary evaluator.
_______________________________________________
That is expected because of the difference in expertise
16. Is the alarm system and its testing 1 1
program entered in the DB? and the difference in the time used for the evaluation.
17. Is PMS self-improvement program 1 1 Both evaluations pronounced all three databases to be
inserted into the DB and is there control good, expert gave an average grade of 4.3 (out of 5).
mechanism for PMS DB Both evaluations concluded that analysed databases
self-improvement program? have all inspected components and well-established
20. Are jobs created and grouped according 2 2 maintenance plan. Main differences of two
to multiplier principle? evaluations are noted in discovered shortcomings of
21. Are all the same type jobs, coming from 2 1 databases and recommendations for the future
different sources, synchronized? actions. The evaluator with ordinary skills using new
22. Are all the same jobs, resulting from 2 1
different requirements (sources), merged? concept concluded that all analysed databases are in
_______________________________________________ order and that there are no areas for the improvement
found during his evaluation. Evaluation performed by
Overall opinion of the expert, based on the expert discovered several areas which need the
extensive inspection of all three databases: improvement, and above all, two of them are
− Databases have an average grade of 4.3, which is classified as serious deficiencies which require
relatively good overall evaluation grade. There is immediate action.
frequent usage of the system by several on-board Comparison of two evaluations showed major
users and several office users, which is by itself discrepancy between evaluation results. Although
good sign. overall evaluation of databases matches, the results of
− Databases appear to have all components, and look evaluations do not. New concept failed to identify any
to be in good order. All components are marked deficiencies which expert discovered in the database
properly and uniquely, according to their and therefore failed in main purpose of the evaluation
shipboard location and markings. The data about which should be to discover problems in the database
the manufacturer, the type and the serial number is and to recommend areas for the improvement.
entered to all relevant items as required.
Maintenance plan is well designed and tuned, all Although there are significant potential savings
jobs required by company policy are included in connected with the new concept (first, the duration of
the DB as well as all jobs required by flag state the evaluation is eight times longer, than expert wages
rules and regulations and by the Class society. Fire are much higher), noted major discrepancies question
detection sensor list has been inserted into the DB the meaning of this evaluation.

741
6 CONCLUSION REFERENCES

Several conclusions about the concept can be 1. Doremus, J., Helfand, G., Liu, C., Donahue, M., Kahan,
formulated, despite relatively small number of A., Shelby, M.: Simpler is better: Predicting consumer
evaluated databases. Testing of the concept of on the vehicle purchases in the short run. Energy Policy. 129,
different PMS verified that the concept is functioning 1404–1415 (2019).
https://doi.org/10.1016/j.enpol.2019.02.051.
and that it can be used on different computerized 2. Geurts, W.J.W.: Faster is Better and Cheaper. INCOSE
systems. Two persons, both fulfilling the condition of International Symposium. 26, 1, 1002–1015 (2016).
average skills, tried the use of the concept and both https://doi.org/10.1002/j.2334-5837.2016.00207.x.
succeeded to evaluate database(s) without any 3. Hoxmeier, J.A.: Typology of database quality factors.
problem. This is confirming that the concept is indeed Software Quality Journal. 7, 3, 179–193 (1998).
simple enough for the use by everybody, as it was https://doi.org/10.1023/A:1008923120973.
intended. Average time for the evaluation was 20-30 4. Jung, H.-J.: The Quantity Data Estimation for Software
minutes which is huge improvement from the time Quality Testing. Journal of the Korea Convergence
needed for the expert evaluation. Short evaluation Society. 8, 10, 37–43 (2017).
https://doi.org/10.15207/JKCS.2017.8.10.037.
time is confirming that another goal of the concept is
5. Kolossa, A., Kopp, B.: Data quality over data quantity in
achieved. computational cognitive neuroscience. Neuroimage. 172,
Comparison of results of the second stage tests 775–785 (2018).
showed a major deficiency of the concept which was https://doi.org/10.1016/j.neuroimage.2018.01.005.
6. McMahon, P., Zhang, T., Dwight, R.A.: Approaches to
noted in the second stage test in evaluations of all Dealing With Missing Data in Railway Asset
three databases. Concept evaluation did not produce Management. IEEE Access. 8, 48177–48194 (2020).
same results as the evaluation performed by the https://doi.org/10.1109/ACCESS.2020.2978902.
expert, furthermore, concept failed to detect serious 7. Pérez-Medina, J.-L., Acosta-Vargas, P., Sampedro, C.,
shortcomings inside databases which were detected Paredes, A., Carrión, M., Santórum, M., Samaniego-
by expert and listed line by line. Although the concept Santillán, L.-P., Maldonado-Garcés, V.-G., Corrales-
showed great prospect, and evaluation of the first Gaitero, C., Ortiz-Carranco, N.-Y.: Using Expert
database was very promising, the discovery of those Evaluation to Assess the Usability of an Educational
shortcomings in the databases changed the outcome. Mobile Game for Cognitive Skills Training. In: Nunes,
I.L. (ed.) Advances in Human Factors and Systems
The concept failed in the primary goal, to perform Interaction. pp. 265–271 Springer International
proper evaluation and determined the outcome. Publishing, Cham (2020).
Although databases evaluated in the second stage test 8. Petrevska, I., Ilic, I., Petrevska, M., Markovic, B.: Kaizen
had larger number of information linked to them, as the Element of Overall Management Quality Control.
there were shortcomings which ordinary skilled Int’l J. Econ. & L. 9, 25, 129 (2019).
person did not detect. Despite good initial results, the 9. Rossi, P.H., Lipsey, M.W., Freeman, H.E.: Evaluation: A
concept failed to produce satisfactory results and Systematic Approach. SAGE Publications, Inc;
further use of the concept is not recommended. 10. Roy-Hubara, N., Sturm, A.: Design methods for the new
database era: a systematic literature review. Software
and Systems Modeling. 19, 2, 297–312 (2020).
https://doi.org/10.1007/s10270-019-00739-8.
11. Stanivuk, T., Stazić, L., Vidović, F., Čobanov, A.: Ship
Planned Maintenance System Data Analysis.
International Journal for Traffic and Transport
Engineering. 10, 4, 432–436 (2020).
https://doi.org/10.7708/ijtte.2020.10(4).02.
12. Stazić, L., Komar, I., Račić, N.: Evaluation Methodology
for Ship’s Planned Maintenance System Database. Trans.
Marit. Sci. 6, 2, 109–116 (2017).
https://doi.org/10.7225/toms.v06.n02.002.
13. Tversky, A., Kahneman, D.: Belief in the law of small
numbers. Psychological Bulletin. 76, 2, 105–110 (1971).
https://doi.org/10.1037/h0031322.

742