Abstract:Abstract: This study discusses the optimization of RAG for a FAQ system in the field of information technology product security certification at BSSN. Although LLM generate reliable responses, they often lack up-to-date…
and domain-specific knowledge, which can be addressed through the RAG approach. This research aims to optimize a domain-specific RAG system by improving embedding performance, enhancing prompt robustness, and increasing retrieval accuracy. The research methods consist of three stages. The first stage involves fine-tuning the bge-m3 embedding model and evaluating its performance using MRR, Recall, and AUC. The second stage applies prompt engineering techniques, namely the SRSM and Autodefense, to mitigate direct-injection and escape-character prompt injection attacks. The third stage evaluates the proposed RAG system using Precision, Recall, and F1-Score metrics against four baseline models. The results of research show that the fine-tuned embedding model achieves higher performance than the original model, with MRR@1 and Recall@1 values of 0.80 and an AUC@100 of 0.7023. In addition, the proposed prompt engineering techniques demonstrate robustness against prompt injection attacks, while the overall RAG system attains a perfect Precision, Recall, and F1-Score of 1.00. In conclusion, the proposed approach effectively enhances retrieval accuracy, embedding quality, and system security, resulting in a more reliable RAG-based FAQ system for information technology product security certification.
Keywords: embedding fine-tuning; large language model; prompt engineering; prompt injection mitigation; retrieval-augmented generation
Abstrak: Studi ini membahas optimasi RAG untuk sistem FAQ di bidang sertifikasi keamanan produk teknologi informasi di BSSN. Meskipun LLM menghasilkan respons yang andal, mereka seringkali kurang memiliki pengetahuan terkini dan spesifik domain, yang dapat diatasi melalui pendekatan RAG. Penelitian ini bertujuan untuk mengoptimalkan sistem RAG spesifik domain dengan meningkatkan kinerja embedding, meningkatkan ketahanan prompt dan meningkatkan akurasi pengambilan. Metode penelitian terdiri dari tiga tahap. Tahap pertama melibatkan fine-tuning model embedding bge-m3 dan mengevaluasi kinerjanya menggunakan Mean Reciprocal Rank (MRR), Recall, dan AUC. Tahap kedua menerapkan teknik rekayasa prompt, yaitu Self- SRSM dan Autodefense, untuk mengurangi serangan direct-injection dan escape-character prompt injection. Tahap ketiga mengevaluasi sistem RAG yang diusulkan menggunakan metrik Presisi, Recall, dan F1-Score terhadap empat model dasar. Hasil penelitian menunjukkan bahwa model embedding yang disempurnakan mencapai kinerja yang lebih tinggi daripada model asli, dengan nilai MRR@1 dan Recall@1 sebesar 0,80 dan AUC@100 sebesar 0,7023. Selain itu, teknik rekayasa prompt yang diusulkan menunjukkan ketahanan terhadap serangan injeksi prompt, sementara sistem RAG secara keseluruhan mencapai Presisi, Recall, dan F1-Score sempurna sebesar 1,00. Kesimpulannya, pendekatan yang diusulkan secara efektif meningkatkan akurasi pengambilan, kualitas embedding dan keamanan sistem, menghasilkan sistem FAQ berbasis RAG yang lebih andal untuk sertifikasi keamanan produk teknologi informasi.
Kata kunci: penyempurnaan embedding; model bahasa besar; rekayasa prompt; mitigasi injeksi prompt; retrieval-augmented generation
Abstract:Abstract: The determination of thesis topics is a fundamental stage in academic research, yet the evaluation process remains predominantly manual and subjective. This reliance on individual lecturer perception often leads…
s to inconsistent feasibility assessments and fails to systematically measure the topic's alignment with strategic needs. This research aims to develop a Decision Support System (DSS) model based on fuzzy logic to assess the feasibility of thesis topics objectively and systematically, focusing on enhancing the relevance of student research. The research method employed the Fuzzy Inference System (FIS) with the Sugeno method. This model was designed through literature review and FGD to establish four criteria (Topic Relevance, Difficulty Level, Idea Novelty, Reference Availability) and 81 rule bases. The model validation results against expert judgment using 15 test data showed a high accuracy rate of 91.31%, with a Mean Absolute Percentage Error (MAPE) value of 8.69%. In conclusion, this DSS model is proven to be valid and consistent, and it can be relied upon as an objective tool to improve the quality and relevance of thesis topics.
Keywords: academic evaluation; decision support system; fuzzy logic; fuzzy sugeno; thesis feasibility
Abstrak: Penentuan topik skripsi merupakan tahapan fundamental dalam penelitian akademik, namun proses evaluasinya hingga kini masih cenderung manual dan subjektif. Ketergantungan pada persepsi dosen secara individu sering kali menyebabkan penilaian kelayakan yang tidak konsisten serta kegagalan dalam mengukur keselarasan topik dengan kebutuhan strategis secara sistematis. Penelitian ini bertujuan mengembangkan model Sistem Pendukung Keputusan (SPK) berbasis logika fuzzy untuk menilai kelayakan topik skripsi secara objektif dan sistematis, dengan fokus pada peningkatan relevansi penelitian mahasiswa. Metode penelitian yang digunakan adalah Fuzzy Inference System (FIS) dengan metode Sugeno. Model ini dirancang melalui tinjauan pustaka dan Focus Group Discussion (FGD) untuk menetapkan empat kriteria (Relevansi Topik, Tingkat Kesulitan, Kebaruan Ide, Ketersediaan Referensi) serta 81 basis aturan. Hasil validasi model terhadap penilaian pakar menggunakan 15 data uji menunjukkan tingkat akurasi yang tinggi yaitu 91,31%, dengan nilai Mean Absolute Percentage Error (MAPE) sebesar 8,69%. Kesimpulannya, model SPK ini terbukti valid dan konsisten, serta dapat diandalkan sebagai alat objektif untuk meningkatkan kualitas dan relevansi topik skripsi.
Kata kunci: evaluasi akademik; sistem pendukung keputusan; logika fuzzy; fuzzy sugeno; kelayakan skripsi
Abstract:Abstract: The rapid expansion of digital learning environments has increased students’ reliance on Learning Management Systems (LMS), including SPOT UPI. However, limited studies have examined the platform’s overall user…
user experience across all User Experience Questionnaire (UEQ) dimensions. This study aims to evaluate the user experience (UX) of SPOT UPI, identify its strengths and weaknesses, and provide recommendations for system improvement. A quantitative-dominant mixed-method design was applied, involving 81 student respondents for the UEQ survey and two participants for follow-up semi-structured interviews selected through purposive sampling. The UEQ data were analyzed to generate mean scores for six UX dimensions, while interview data were thematically analyzed to support the interpretation of quantitative findings. The results indicate that Perspicuity (1.05) and Efficiency (0.78) achieved the highest scores, reflecting adequate clarity and functionality. In Contrast, Stimulation (0.50) and Novelty (-0.15) were the lowest, indicating limited engagement and innovation. Overall, pragmatic quality (0.84) outperformed hedonic quality (0.17), suggesting that users value functionality more than enjoyment. In conclusion, SPOT UPI is generally usable but lacks aesthetic appeal, emotional engagement, and innovative features, highlighting the need for interface redesign and performance optimization to enhance the overall learning experience.
Keywords: learning management system; user experience; user experience questionnaire
Abstrak: Perkembangan pembelajaran digital membuat mahasiswa semakin bergantung pada Learning Management System (LMS), termasuk SPOT UPI. Meski digunakan secara luas, evaluasi pengalaman pengguna secara komprehensif berdasarkan seluruh dimensi User Experience Questionnaire (UEQ) masih belum banyak dilakukan. Penelitian ini bertujuan untuk mengevaluasi user experience (UX) pada SPOT UPI, mengidentifikasi keunggulan dan kelemahannya, serta memberikan rekomendasi perbaikan sistem. Penelitian menggunakan desain penelitian mixed-method dominan kuantitatif, melibatkan 81 responden pada survei UEQ dan dua partisipan pada wawancara semi-terstruktur yang dipilih melalui purposive sampling. Data UEQ dianalisis untuk memperoleh nilai rata-rata pada enam dimensi UX, sedangkan data wawancara dianalisis secara tematik untuk memperkaya interpretasi temuan kuantitatif. Hasil menunjukkan bahwa Perspicuity (1,05) dan Efficiency (0,78) menjadi dimensi dengan skor tertinggi, mencerminkan bahwa SPOT UPI mudah dipahami dan cukup membantu dalam menyelesaikan tugas. Sebaliknya, Stimulation (0,50) dan Novelty (-0,15) memperoleh skor terendah, menandakan rendahnya tingkat keterlibatan dan inovasi yang dirasakan pengguna. Secara keseluruhan, pragmatic quality (0,84) lebih tinggi dibandingkan hedonic quality (0,17), menunjukkan bahwa pengguna lebih mengutamakan aspek fungsional daripada kenyamanan emosional. Temuan tersebut mengindikasikan bahwa SPOT UPI sudah layak digunakan secara fungsional, tetapi masih memerlukan peningkatan pada interface, pengalaman visual, dan fitur inovatif agar dapat memberikan pengalaman belajar digital yang lebih menarik dan optimal.
Kata kunci: learning management system; pengalaman pengguna; user experience questionnaire
Abstract:Abstract: SMP Muhammadiyah 5 Samarinda still relies on manual evaluation with limited data analysis tools in predicting student academic achievement. This study aims develop a system for predicting the learning achievement…
nt of students at SMP Muhammadiyah 5 Samarinda using the Naive Bayes classification method. The dataset used consists of 192 student exam scores covering academic scores, attendance, parents’ education and income, and living conditions as independent variables, while the dependent variable is the achievement label (achieved or not achieved). The preprocessing stage includes label normalization, feature selection, and median imputation to handle missing data. The dataset was divided into 75% training data and 25%. The model was implemented as a pipeline consisting of a median imputer and a Gaussian Naive Bayes classifier. The evaluation results showed that the model achieved an accuracy of 79.2%, with a perfect recall value (1.00) in the high-achieving class and (0.64) in the low-achieving class. This shows that the model is quite effective in identifying high-achieving students. The trained model was then integrated into a Flask-based web application, which enables online predictions through a simple form interface, facilitating contextual interpretation. This system is expected to assist in educational decision-making by helping teachers identify students’ achievement levels early on and design more targeted learning interventions.
Keywords: academic performance; educational data mining; naive bayes; prediction system; student achievement
Abstrak: SMP Muhammadiyah 5 Samarinda masih bergantung pada evaluasi manual dengan alat analisis data terbatas dalam melakukan prediksi prestasi akademik siswa. Penelitian ini bertujuan mengembangkan sistem prediksi prestasi belajar siswa SMP Muhammadiyah 5 Samarinda menggunakan metode klasifikasi Naive Bayes. Dataset yang digunakan terdiri atas 192 data nilai ujian siswa yang mencakup skor akademik, kehadiran, pendidikan dan pendapatan orang tua, serta kondisi tempat tinggal sebagai variabel independen, sedangkan variabel dependen berupa label prestasi (berprestasi atau tidak berprestasi). Tahap preprocessing meliputi normalisasi label, seleksi fitur, serta imputasi median untuk menangani data yang hilang. Dataset dibagi menjadi 75% data latih dan 25%. Model diimplementasikan dalam bentuk pipeline yang terdiri atas median imputer dan Gaussian Naive Bayes classifier. Hasil evaluasi menunjukkan bahwa model mencapai akurasi sebesar 79,2%, dengan nilai recall sempurna (1,00) pada kelas berprestasi dan lebih rendah (0,64) pada kelas tidak berprestasi. Hal ini menunjukkan bahwa model cukup efektif dalam mengidentifikasi siswa berprestasi. Model yang telah dilatih kemudian diintegrasikan ke dalam aplikasi web berbasis Flask, yang memungkinkan prediksi secara daring melalui antarmuka formulir sederhana untuk mendukung interpretasi kontekstual. Sistem ini diharapkan dapat membantu untuk pengambilan keputusan dalam pendidikan dengan membantu guru mengidentifikasi tingkat prestasi siswa sejak dini dan merancang intervensi pembelajaran yang lebih terarah.
Kata kunci: prestasi akademik; penambangan data Pendidikan; naive bayes; sistem prediksi; prestasi siswa
Abstract:Abstract: The increasing use of digital banking applications has led to the need for a deeper understanding of user perceptions, especially through aspect-based sentiment analysis. This study aims to classify the sentiment…
nt of SeaBank app users by focusing on four main aspects: learnability, efficiency, technical issues or errors, and satisfaction. Review data totaling 1,971 comments were collected from the Google Play Store and labeled with sentiments based on the scores (ratings) given by users. The CRISP-DM approach serves as the methodological framework for this study, which includes five classification algorithms: Naïve Bayes, Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Decision Tree, and Random Forest. The evaluation results show that the SVM algorithm provides the best performance with the highest average value of the four aspects achieving accuracy of 93.91%, Precision of 91.16%, recall of 97.96% and F1-Measure of 94.33%. According to the research findings, the Support Vector Machine (SVM) algorithm provides the best performance when performing aspect-based sentiment analysis on text data from digital banking application reviews. The findings are expected to serve as a reference for the development of automated evaluation systems that rely on user opinions as the basis for decision making.
Keywords: aspects; CRISP-DM; digital Banking; seabank; sentiment analysis
Abstrak: Peningkatan pemakaian aplikasi perbankan digital mendorong perlunya pemahaman yang lebih dalam mengenai persepsi pengguna, terutama melalui analisis sentimen berbasis aspek. Penelitian ini bertujuan untuk mengklasifikasikan sentimen pengguna aplikasi SeaBank dengan berfokus pada empat aspek utama: kemudahan dipelajari (learnability), efisiensi penggunaan (efficiency), kendala atau kesalahan teknis (error), serta tingkat kepuasan (satisfaction). Data ulasan berjumlah 1.971 komentar dikumpulkan dari Google Play Store dan diberi label sentimen berdasarkan skor (rating) yang diberikan oleh pengguna. Pendekatan CRISP-DM berfungsi sebagai kerangka metodologis untuk penelitian ini, yang mencakup lima algoritma klasifikasi: Naïve Bayes, Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Decision Tree, dan Random Forest. Hasil evaluasi menunjukkan bahwa algoritma SVM memberikan performa terbaik dengan nilai rata-rata dari ke empat aspek tertinggi yang mencapai accuracy sebesar 93.91%, Precision sebesar 91.16%, recall sebesar 97.96% dan F1-Measure sebesar 94.33%. Menurut temuan penelitian, algoritma Support Vector Machine (SVM) memberikan kinerja terbaik saat melakukan analisis sentimen berbasis aspek pada data teks dari ulasan aplikasi Seabank. Temuan ini diharapkan dapat menjadi referensi bagi pengembangan sistem evaluasi otomatis yang mengandalkan opini pengguna sebagai dasar pengambilan keputusan.
Kata kunci: Analisis Sentimen, Aspek, Bank Digital, SeaBank, CRISP-DM
Abstract:Abstract: Stroke is one of the leading causes of death and disability in various parts of the world, including in Indonesia. Along with the development of digital technology, the use of Machine Learning in the health sector…
tor is growing, one of which is in an effort to predict the occurrence of stroke. This study aims to implement the Logistic Regression algorithm in predicting the likelihood of a person having a stroke based on data from the Brain Stroke dataset. The research process includes data preprocessing (missing value handling, normalization, and label encoding), dividing the data into 80% training data and 20% test data, as well as model training. The model was then evaluated using several measures such as accuracy, precision, recall, F1-score, and ROC-AUC, as well as a confusion matrix. The results of the study showed that Logistic Regression was able to provide stroke classification results with an accuracy of 82.4%, precision of 80.1%, recall of 78.6%, F1-score of 79.3%, and a ROC-AUC value of 0.87. Then, the model is integrated into applications that use Streamlit, so it can be used interactively to predict stroke risk in new data. The results of this study show that the combination of Machine Learning and web-based applications has the potential to support efforts to detect early stroke risk.
Keywords: logistic regression; machine learning; prediction; streamlit; stroke.
Abstrak: Stroke adalah salah satu penyebab utama kematian dan kecacatan di berbagai belahan dunia, termasuk di Indonesia. Seiring perkembangan teknologi digital, penggunaan Machine Learning dalam bidang kesehatan semakin berkembang, salah satunya dalam upaya memprediksi terjadinya penyakit stroke. Penelitian ini bertujuan untuk mengimplementasikan algoritma Logistic Regression dalam memprediksi kemungkinan seseorang mengalami stroke berdasarkan data dari dataset Brain Stroke. Proses penelitian meliputi preprocessing data (penanganan missing value, normalisasi, dan label encoding), membagi data menjadi 80% data latih dan 20% data uji, serta pelatihan model. Model kemudian dievaluasi menggunakan beberapa ukuran seperti akurasi, precision, recall, F1-score, dan ROC-AUC, serta confusion matrix. Hasil penelitian menunjukkan bahwa Logistic Regression mampu memberikan hasil klasifikasi penyakit stroke dengan akurasi sebesar 82,4%, precision 80,1%, recall 78,6%, F1-score 79,3%, dan nilai ROC-AUC sebesar 0,87. Kemudian, model tersebut diintegrasikan ke dalam aplikasi yang menggunakan Streamlit, sehingga dapat digunakan secara interaktif untuk memprediksi risiko stroke pada data baru. Hasil penelitian ini menunjukkan bahwa kombinasi Machine Learning dan aplikasi berbasis web berpotensi mendukung upaya deteksi dini risiko stroke.
Kata kunci: logistic regression; machine learning; prediksi; streamlit; stroke.
Abstract:Abstract: YouTube is one of the most popular video streaming platforms, but it has constraints that can cause problems when clients have difficulty finding content according to their wishes. The main objective of this study…
udy is to increase user capacity in viewing content specifically in the field of women's empowerment. By using content-based filtering techniques, the system will analyze user preferences and interests through recommendations for women's empowerment content. The data source is via the YouTube API and is analyzed using PHP programming content-based filtering techniques. The system's recommendations provide a list of women's empowerment content with a user request display. The results of the research evaluation obtained a precision value of 62%, meaning that the recommendations match the topic being searched for, namely women's empowerment. The recall value of 84% indicates that the system has succeeded in finding relations from the database. The f1-score value of 72% indicates that there is a balance between precision and recall, meaning that a system is needed that is not only accurate but also complete. While the cosine value shows a score of 0.7071 approaching the maximum value (1.0). The recommendation of the content-based filtering method produces quite effective women's empowerment content.
Keywords: content-based filtering, recommendations, women Empowerment, youtube
Abstract:Abstract: Ineffective drug demand management can lead to problems such as imbalanced drug distribution, excess stock, or shortages in community health centers. To address this, data mining can be utilized to support the…
planning and control process of drug inventory. Clustering techniques were chosen because they are able to group drug data based on certain characteristics, thus identifying stable and unstable drug supply patterns. This study aims to group drug data at Simpang Kawat Community Health Center in Jambi City, which can be used as a reference in planning drug needs in the next period. Data grouping is divided into three categories: slow-moving, medium-moving, and fast-moving. The research data includes attributes of drug name, initial stock, receipt, inventory, usage, and final stock, with a total of 1758 data sets, which were processed using the CRISP-DM framework through the RapidMiner application. Cluster quality evaluation was carried out using the Davies-Bouldin Index (DBI). The results showed that the K-Means algorithm obtained a DBI value of 0.175, smaller than K-Medoids which obtained a value of 0.354. Because a smaller DBI value indicates better cluster quality, K-Means provides more optimal clustering results than K-Medoids. Through these clustering results, community health centers can utilize drug cluster information to support more efficient drug procurement planning, as well as reduce the risk of excess or shortage of stock.
Keywords: data mining; clustering; k-means; k-medoids; davies-bouldin index
Abstract:Abstract: In the era of the Internet of Things (IoT), cyber threats are increasingly complex and dynamic, thus demanding an adaptive and intelligent network security system. This study proposes a Convolutional Neural Network…
work (CNN)-based Intrusion Detection System (IDS) implemented through a Federated Learning (FL) approach in a Non-Independent and Identically Distributed (Non-IID) data environment. This approach allows the model to be trained in a distributed manner across multiple IoT devices without having to collect sensitive data to a central server, thereby maintaining data privacy while increasing the efficiency of the training process. The experiment used the CIC IoT 2023 dataset, which represents various modern IoT network traffic patterns. The results show that the proposed CNN–FL model achieves an overall accuracy of 0.99, with excellent performance in detecting various types of network traffic. The model obtains a perfect recall value (1.00) for normal traffic (Benign), as well as a very high F1-score for DDoS (0.99) and DoS (0.99) attacks. Stable and consistent performance across all five federation rounds demonstrates that this approach is a reliable, efficient, and accurate solution for detecting threats in distributed and privacy-preserving IoT networks.
Keywords: cnn; federated_learning; ids; non-iid; ciciot2023
Abstrak: Dalam era Internet of Things (IoT), ancaman siber semakin kompleks dan dinamis, sehingga menuntut sistem keamanan jaringan yang adaptif dan cerdas. Penelitian ini mengusulkan Intrusion Detection System (IDS) berbasis Convolutional Neural Network (CNN) yang diterapkan melalui pendekatan Federated Learning (FL) pada lingkungan data yang bersifat Non-Independent and Identically Distributed (Non-IID). Pendekatan ini memungkinkan model dilatih secara terdistribusi di berbagai perangkat IoT tanpa harus mengumpulkan data sensitif ke server pusat, sehingga mampu menjaga privasi data sekaligus meningkatkan efisiensi proses pelatihan. Eksperimen menggunakan dataset CIC IoT 2023, yang merepresentasikan berbagai pola lalu lintas jaringan IoT modern. Hasil penelitian menunjukkan bahwa model CNN–FL yang diusulkan mencapai akurasi keseluruhan sebesar 0.99, dengan performa yang sangat baik dalam mendeteksi berbagai jenis lalu lintas jaringan. Model memperoleh nilai recall sempurna (1.00) untuk lalu lintas normal (Benign), serta nilai F1-score yang sangat tinggi untuk serangan DDoS (0.99) dan DoS (0.99). Kinerja yang stabil dan konsisten di seluruh lima putaran federasi membuktikan bahwa pendekatan ini merupakan solusi yang andal, efisien, dan akurat untuk mendeteksi ancaman pada jaringan IoT yang bersifat terdistribusi dan menjaga privasi (privacy-preserving).
Kata kunci: cnn; federated_learning; ids; non-iid; ciciot2023
Abstract: Abstract: The Ombudsman of the Republic of Indonesia is an institution tasked with supervising the administration of public services and handling community complaint reports related to allegations of maladministration.…
on. The purpose of this research is to create a decision support system using the Analytic Hierarchy Process (AHP) method, which facilitates the determination of priority handling of community complaint reports at the Ombudsman of the Republic of Indonesia Bengkulu Representation. This decision support system is built on a web-based platform using PHP programming language with a MySQL database that can be accessed offline by the admin of the Ombudsman. With the existence of this priority recommendation, it is expected that work will become more effective and efficient, as resources can be focused on reports that most need attention. Based on the test data used, which consists of 12 Community Complaint Reports from July 2024, it was found that the priority handling recommendations for community complaint reports were derived from 3 reports with the highest final AHP values. The recommended priority handling reports are registration number 0021/LM/VII/2024/BKL with a final AHP value of 2.074, registration number 0020/LM/VII/2024/BKL with a final AHP value of 1.964, and registration number 0018/LM/VII/2024/BKL with a final AHP value of 1.866.
Keywords: decision support system; priority recommendation; public complaint report; AHP Method (analytic hierarchy process method)