Improving Mobile JKN: An Analysis of

Problematic Features and Prototype Solution

Fathi Al Adha Hylmi Kreshnayogi Dava Berliansyach Varisan Falih Ramadhan
22/492195/PA/21088 22/496686/PA/21352 22/497143/PA/21400

Melvin Cahyadi Tirtayasa Raditya Aulia

22/492345/PA/21107 22/492163/PA/21076

Abstract—Mobile JKN is a mobile application developed by Beyond these performance metrics, our analysis extends to a
BPJS Kesehatan to facilitate participants in accessing various comprehensive evaluation of the application's functionalities
healthcare services. The application, designed to facilitate to discern potential factors negatively impacting user
access to healthcare services, has faced several usability satisfaction. This involves scrutinizing user interactions and
challenges impacting its effectiveness and user satisfaction. experiences meticulously to identify both overt deficiencies
This report presents an analysis of the problematic features and subtleties that may have eluded prior scrutiny. By
identified in the current version of Mobile JKN. adopting this approach, we endeavor to offer a nuanced
perspective essential for informed decision-making and
Keywords—Mobile JKN, mobile application, health,
healthcare, reviews, improvement, feature, OTP.
targeted enhancements.
Moreover, our analysis delves into the underlying
reasons behind user dissatisfaction, evaluating not only the
I. INTRODUCTION application's functionality but also its usability, accessibility,
and overall user experience. Through this holistic approach,
A mobile application constantly undergoes maintenance
we aim to provide a comprehensive understanding of the
and updates to improve its quality. Before the update, the
various factors influencing user satisfaction, thereby
developers go through a complex process of assessing the
illuminating both strengths and weaknesses. Equipped with
current features of the application, determining the parts to
this insight, stakeholders can devise strategic interventions,
be transformed, removed or upgraded. Mobile JKN,
prioritizing improvements with the greatest potential to
developed by BPJS Kesehatan, is no exception. Despite its
enhance user satisfaction.
aim of facilitating access to healthcare services for users, it
has encountered several problems in its features that hinder Ultimately, this paper serves as a strategic guide for
its effectiveness and overall user satisfaction. refining the application, ensuring its alignment with user
expectations and bolstering its competitive standing in the
Mobile JKN is a mobile application developed by BPJS
digital domain. By systematically investigating these areas,
Kesehatan, the Health Social Security Administration Body
the study aims to provide a comprehensive comparison of
of Indonesia. The primary purpose of the Mobile JKN app is
the two server architectures and their impact on mobile
to facilitate participants' access to various healthcare
application performance, offering valuable insights for
services provided under the Indonesian National Health
developers and decision-makers.
Insurance (JKN) program. This app is designed to help the
interaction between users and the healthcare system,
offering a range of functionalities that enhance the overall
healthcare experience. This report will present a III. METHODOLOGY
comprehensive analysis of the current version of Mobile To efficiently and effectively obtain the application’s
JKN, analyzing the issues affecting its usability. By ratings and reviews to identify the most mentioned
examining these issues, we propose solutions to address complaints, we decided that we would extract all the
them. Our approach is to develop a prototype of one its reviews in detail. To achieve this goal, we utilized python,
features to improve said feature from the original data mining alongside the concept of natural language. After
application. we did the content analysis we developed a prototype of one
of the features to present our solution for the problem.
A. Web Scraping
II. OBJECTIVE
The primary aim of this paper is to conduct an in-depth To begin, we accessed the Google Play page of the
examination of mobile applications to test the hypothesis Mobile JKN application and employed web scraping
that those developed and hosted using Firebase servers techniques to collect a comprehensive dataset comprising
demonstrate superior performance in terms of speed both positive and negative user reviews. Due to our focus on
compared to those developed and hosted using negative comments, we filtered the dataset to include only
government-provisioned physical servers. In addition to this negative reviews for subsequent analysis.
primary focus, we aim to discern potential factors that may
impact users' satisfaction negatively, recognizing that in the
contemporary digital landscape, user satisfaction is a pivotal
determinant of an application's success.
 In this code snippet, various parameters are specified to
Python customize the review retrieval process. The
from google_play_scraper import app sleep_milliseconds parameter determines the interval
between consecutive requests to the Google Play Store;
import pandas as pd here, it is set to 0 milliseconds, indicating no delay between
requests. The lang parameter specifies the language of the
import numpy as np reviews to be retrieved, with 'en' representing English. The
country parameter indicates the country from which reviews
from google_play_scraper import Sort, are to be collected; in this case, it is set to 'us' for the United
reviews_all States. Lastly, the sort parameter determines the sorting
order of the reviews, with [Link] indicating that
reviews are sorted by newest first.
us_reviews = reviews_all(
Once the reviews are retrieved, they are stored in a
'[Link]',
DataFrame named df_busu. Initially, all the reviews are
sleep_milliseconds=0, # defaults
stored in a single column named 'review'. However, the
to 0
.join() method is then used to split the review data into
lang='en', # defaults to 'en'
individual columns, such as 'reviewId', 'userName',
country='id', # defaults to 'us'
'userImage', 'content', 'score', 'thumbsUpCount', and
sort=[Link], # defaults to 'reviewCreatedVersion', based on the attributes of each
Sort.MOST_RELEVANT review.
)
Finally, the resulting DataFrame df_busu is printed,
df_busu = displaying the user reviews along with associated metadata
[Link]([Link](us_reviews),colu such as the review ID, user name, review content, rating
mns=['review']) score, number of thumbs up, and review creation version.

B. Text Mining for Words

df_busu =
Text mining, also known as text data mining or text
df_busu.join([Link](df_busu.pop(
analytics, is a method of extracting high-quality
'review').tolist()))
information from various written sources like websites,
books, emails, reviews, and articles. In this study, we
df_busu.to_csv('[Link]',
applied text mining to extract the words and their
index=False)
frequency on the applications’ reviews. Below is the
script coded for the text mining and the result of the
df_busu program[2].

Python
[3]
import csv
import collections
import re

# Create a counter to store word

frequencies
word_freq =
[Link]()

Python # Specify the path to your CSV

df.to_csv('[Link]', index=False) file
file_path =
"C:/Users/hylmi/OneDrive - UGM
This Python code utilizes the google_play_scraper 365/Coding/[Link]"
library to retrieve user reviews for the application with
package name '[Link]' from the Google Play # Open the CSV file and read the
Store. The reviews_all function is employed to fetch all reviews
reviews for the specified application.
 The script starts by loading the necessary libraries for
with open(file_path, 'r', managing word frequency (collections), handling CSV files
(csv), and preparing text using regular expressions (re). The
newline='', encoding='utf-8') as
file location for the CSV dataset is defined, and a Counter
file: object to track word frequencies is initialized. The script
reader = reads the CSV file by treating each row as a dictionary using
[Link](file) [Link]. It removes reviews that have a rating of
three or lower (negative reviews) and preprocesses the
content by converting any non-alphabetic characters to
# Go through each review
lowercase while looping over each review. Word
for row in reader: frequencies are updated in the counter when the text has
# Check if the rating is been divided into individual words, taking into account
3 or less terms with three or more alphabets. This method makes it
if int(row['score']) <= easier to extract insightful information from the dataset,
especially when it comes to figuring out recurring themes or
3:
problems in the unfavorable evaluations.
# Remove
non-alphabetic characters and C. Text Mining for Sentences
convert to lower case
words =
[Link]('[^a-zA-Z]', ' ', Python
row['content']).lower().split() import csv
import collections
import re
# Update word
import nltk
frequencies only for words with [Link]('punkt')
at least 3 alphabets
for word in words: # Create a counter to store sentence
if len(word) >= frequencies
3: sentence_freq = [Link]()

# Specify the path to your CSV file

word_freq[word] += 1 file_path = "C:/Users/hylmi/OneDrive -
UGM 365/Coding/[Link]"
# Print the most common words
for word, freq in # Specify your keywords
word_freq.most_common(10): keywords = ['tidak', 'bisa',
'aplikasi'] # replace with your
print(f'Word: {word},
keywords
Frequency: {freq}')
# Open the CSV file and read the
reviews
with open(file_path, 'r', newline='',
encoding='utf-8') as file:
reader = [Link](file)

# Go through each review

for row in reader:
# Check if the rating is 3 or
less
if int(row['score']) <= 3:
# Split the review into
sentences
sentences =
nltk.sent_tokenize(row['content'])

# Go through each sentence

 Finally, the code prints the ten most common sentences
for sentence in sentences: along with their frequencies, providing insights into the
# Convert to lower prevalent issues or concerns expressed by users in the
case and split into words negative reviews.
words =
[Link]('[^a-zA-Z]', ' ', Output:
sentence).lower().split() Sentence: Aplikasi kesehatan malah maintenance di jam
kerja, pas butuh malah gk bisa di akses, KACAU!!!,
# If any of the words Frequency: 1
in the sentence are in your keywords, Sentence: Aplikasi kentang sering eror, kendala, ga bisa
update the sentence frequency login., Frequency: 1
if any(word in words
Sentence: Aplikasi pemerintah tapi sampah banget lol,
Frequency: 1
for word in keywords):
Sentence: ga jelas banget, mau daftar aja eror terus atau ga
pelayanan tidak ditemukan., Frequency: 1
sentence_freq[sentence] += 1
Sentence: Aplikasi pemerintah tapi mata duitan , mau
ngirim kode verifikasi aja kena pulsa 1000 , udah gitu force
# Print the most common sentences
close terus , kalah sama game online , game online pb aja
for sentence, freq in verifikasi ga pakai pulsa , lawak, Frequency: 1
sentence_freq.most_common(10): Sentence: aplikasi apaan ini!, Frequency: 1
print(f'Sentence: {sentence}, Sentence: daftar gak bisa, login gak bisa, lupa password gak
Frequency: {freq}') bisa, cek nomor tlp sudah dipakai pengguna lain hadeh,
Frequency: 1
Firstly, the code imports necessary libraries including Sentence: Tapi, setelah melakukan pendaftaran antrian di
csv, collections, re (regular expression), and nltk (Natural hari ini..untuk saya melakukan pendaftaran antrian buat
Language Toolkit). The [Link]('punkt') line ensures besoknya ga bisa, karena antrian sebelumnya masih ada dan
that the necessary NLTK data required for sentence tidak bisa dihilangkan jadi saya masih belum bisa
tokenization is available. melakukan ambil antrian terbaru untuk besok., Frequency: 1
Sentence: Aplikasi ribet, Frequency: 1
Next, a counter named sentence_freq from the Sentence: aplikasi ga jelas, error mulu, Frequency: 1
collections module is initialized to store the frequencies of
sentences containing the specified keywords.
D. Visualization
The path to the CSV file containing user reviews is
specified in the file_path variable. Additionally, a list of
keywords, such as ['tidak', 'bisa', 'aplikasi'], is defined to Python
identify relevant sentences within the reviews. These from wordcloud import WordCloud
keywords are crucial for filtering out sentences that express import [Link] as plt
dissatisfaction or issues with the application.
# Combine all reviews into a single
The code then opens the CSV file using a with statement string
and reads its contents using the [Link] function. It all_reviews_text = '
iterates through each row of the CSV file, extracting the
'.join(df_busu['content'])
review content and the associated rating score.

For each review with a rating score of 3 or less

exclude_words = ['saya', 'aplikasi',
(indicating a negative review), the review content is split
'dan', 'yang', 'app',
into individual sentences using the nltk.sent_tokenize()
function. This ensures that each sentence is treated as a 'ada','ini','di','untuk','sudah','tida
separate entity for analysis. k','bisa','good','sangat']

Within each sentence, the code converts all characters to def filter_words(text):
lowercase and splits the text into words, removing any words = [Link]()
non-alphabetic characters in the process. This step facilitates filtered_words = [word for word in
uniform comparison and keyword matching. words if [Link]() not in
exclude_words]
The code then checks if any of the words in the sentence return ' '.join(filtered_words)
match the predefined keywords. If at least one keyword is
found in the sentence, the frequency count of that sentence
is incremented in the sentence_freq counter.
 analysis and might not capture the full context of each
review. For a more accurate analysis, more advanced natural
filtered_reviews_text = language processing techniques or libraries might be
filter_words(all_reviews_text) necessary. Also, be respectful and make sure you’re not
violating any terms of service.
# Generate word cloud
wordcloud = WordCloud(width=800,
V. DISCUSSION
height=400,
background_color='white').generate(fil The findings from the analysis of user reviews shed light on
tered_reviews_text) critical areas of concern within the mobile application
ecosystem, particularly regarding user experience and
# Plot word cloud functionality. Three predominant themes emerged from the
[Link](figsize=(10, 5)) reviews, namely: one-time passwords, login problems, and
[Link](wordcloud,
performance issues. These issues not only undermine the
usability and reliability of the application but also pose
interpolation='bilinear')
significant challenges for user engagement and satisfaction.
[Link]('off')
[Link]('Word Cloud of Most Common
A. Plan
Words in Reviews')
[Link]()
To address these key issues, a comprehensive improvement
of the app's backend infrastructure was undertaken, focusing
primarily on the login system. The improvements included:

● Enhanced OTP System: The OTP generation and

delivery system was redesigned to ensure faster
and more reliable performance. This involved the
usage of an OTP service such as FireBase
Authentication.
● Improved Login Mechanism: The login process
was streamlined by reducing the requirements for
the login system. This included the removal of the
requirement for email and NIK to reduce latency
and prevent bottlenecks during peak times.
● Performance Optimization: Overall app
performance was enhanced by reworking the login
system for faster query responses and better
IV. RESULT handling of concurrent transactions.
Based on the analysis of the reviews with a rating of 3 or
less, the most frequently used words with at least three
alphabets are predominantly in Indonesian. The word
“tidak” (meaning “no” or “not” in English) is the most B. Implementation
common, appearing 189 times. This is followed by “bisa” In the project progress, we created a prototype application
(meaning “can” or “able to” in English) which appears 182 using firebase that allowed OTP sending after the user
times, and “aplikasi” (meaning “application” in English) inputted the mobile phone number and did the verification.
which appears 169 times. The frequent appearance of these
words suggests that users might be expressing difficulties or
issues with the application. The word “login” also appears
frequently (130 times), indicating that users might be having
trouble logging in to the application.

When we look at the most common sentences that contain at

least one of the three most frequent words (“tidak”, “bisa”,
“aplikasi”), we see similar themes. The sentence “Tidak bisa
login.” (meaning “Cannot log in.” in English) is the most
common, appearing 3 times.

In conclusion, the analysis of the reviews suggests that users

are experiencing issues with logging in, otp, and
performance issues. These insights could be valuable for the
application’s developers to improve the user experience.
However, please note that this is a basic form of text
 based on the reviews acquired. During testing, we have not
had an OTP failure. So clearly, we have a more reliable OTP
system.
VII. CONCLUSION

Our study indicates that while using a high-performance

server like Firebase can significantly enhance the speed and
reliability of mobile applications, the overall system flow
plays an even more critical role in user satisfaction. The
issues identified in the Mobile JKN app, such as login
difficulties and OTP verification problems, highlight that
optimizing backend infrastructure alone is insufficient. A
well-designed, intuitive system flow that prioritizes user
experience and addresses common pain points is essential
for achieving high user satisfaction. Thus, combining robust
server performance with a thoughtfully designed user
interface and seamless functionality ensures a superior app
experience. This holistic approach not only resolves
immediate technical issues but also fosters long-term user
engagement and satisfaction.

REFERENCES
Since our primary focus was on enhancing the login system,
specifically addressing the OTP functionality and [1] “Mobile JKN - Aplikasi di Google Play.” n.d.
performance issues, we concentrated solely on these areas. Google Play. Accessed June 9, 2024.
Consequently, we did not redesign the entire application [Link]
interface. Instead, we focused exclusively on the login user [Link]&hl=id.
interface to effectively tackle the critical issues identified in
our analysis. [2] IBM. n.d. “What Is Text Mining?” IBM. Accessed
March 31, 2024.
[Link]

[3] kundi, gagandeep. 2021. “How to Scrape Google

Play Reviews in 4 simple steps using Python.”
LinkedIn.
[Link]
e-play-reviews-4-simple-steps-using-python-kundi/

[4] Mueller, A. (2020). WordCloud for python

documentation¶. WordCloud for Python
documentation - wordcloud 1.8.1 documentation.
[Link]

Based on the pictures above, we successfully receive the

OTP code after inputting the phone number on the
application.. The OTP was sent into the inserted phone
number. The problem with the OTP application that we
created was that the current state of the OTP page cannot be
saved when visiting the message to check the OTP, therefore
the phone number has to be typed again to receive the new
OTP.

VI. EVALUATION
Since we cannot do large-scale testing on the Mobile JKN
application, we can look at the reviews as a survey data to
see the percentage of people who are having trouble with
the OTP. Based on the total number of extracted reviews
through content analysis, which are the most recent 199
reviews, we obtain 11 reviews related to the OTP problem.
This indicates that the rate of the OTP failure was 5.5%