TEXT CLASSIFICATION OF TWITTER OPINION RELATED TO PERMENDIKBUD 30/2021 USING BIDIRECTIONAL LSTM

Article History: During the COVID-19 outbreak, sexual violence in Indonesia has risen. Sexual abuse is prevalent even within the realm of education. Many incidents of sexual assault are reported within the higher education sector. The Ministry of Education, Culture, Research, and Technology published Decree of the Minister of Education and Culture Number 30/2021 (Permendikbud 30/2021) on the Prevention and Handling of Sexual Violence in Higher Education in an effort to prevent sexual violence on campus. This regulation's issuance has become a popular topic of discussion on social media. Twitter is one of the social media platforms where opinions are expressed. The publication of Permendikbud 30/2021 elicited a variety of views, from those who supported the rule to those who did not. This study's objective is to categorize tweets about Permendikbud 30/2021. Bidirectional LSTM (BiLSTM) was utilized to classify data in this study. The accuracy values are 87%, the precision values are 82%, and the recall values are 96% based on the findings of the analysis comparing training data of 80% to testing data of 20%.


INTRODUCTION
According to the Decree of the Minister of Education and Culture Number 30/2021 (Permendikbud 30/2021), sexual violence is any act of humiliating, harassing, and/or attacking a person's body and/or reproductive function, due to an imbalance of power and/or gender relations, which results in or can result in psychological suffering and/or interferes with a person's reproductive health and the opportunity to pursue higher education safely and optimally [1]. Sexual assault is not limited to rape alone. In addition, every activity that leads to sexuality through coercion harms someone. Sexual assault can occur anywhere, including at home, in the surrounding environment, and even in the realm of education, which is a dangerous place to learn. Sexual violence can occur at all educational levels, from early childhood schooling through higher education. The highest number of reports of violence against women were received by the Komnas Perempuan from tertiary institutions, which ranked first [2].
Not all instances of sexual violence on campuses are disclosed. From a survey conducted by the Ministry of Education and Culture in 2020 on lecturers, it was found that 77% of sexual violence occurred within universities, with 63% of them not reporting it [3]. For different reasons, some victims choose not to disclose or complain about sexual violence. The absence of rules that safeguard victims is one of them. Also contributing to campus confusion is the absence of rules and norms guiding the treatment of sexual violence on university campuses. Therefore, the Ministry of Education, Culture, Research, and Technology enacted Permendikbud 30/2021 on the Prevention and Handling of Sexual Violence in Higher Education in an effort to prevent sexual violence in universities.
The publication of Permendikbud 30/2021 on the Prevention and Handling of Sexual Violence has been a trending topic on Twitter. Twitter is a microblogging social media that Indonesians frequently utilize to communicate their thoughts via chirps, also known as tweets [4]. The community's responses to this vary. Multiple parties agree with and support the regulation to prevent sexual violence at school and foster a sense of safety on campus. However, many disagree with this rule and believe it could legalize adultery.
From tweets related to Permendikbud 30/2021, it can be classified into two categories, namely pros and cons. Classification is a process of find a model or function that explains or distinguishes concepts and data classes, with the aim of being able to estimate the class of an object whose class is unknown [5]. There are several classification methods. One method that can be used for classification is bidirectional LSTM (BiLSTM).
BiLSTM is a deep learning method that is an extension of long short-term memory (LSTM). BiLSTM has two layers whose processes are reversed, namely the lower layer that moves forward (forward) and understands and processes from the first word to the last word, and the layer above it that moves backward (backward), namely understanding and processing from the last word to the first word [6]. The advantage of this method is that it can understand the context of the sentence and represent the text better because it has a two-way layer that can take perspective from the previous word and the leading word.
This study aims to classify tweets that support (pro) and reject (contra) the regulation of the Minister of Education and Culture Number 30 of 2021 concerning the Prevention and Handling of Sexual Violence (PPKS) using the Bidirectional Long Short-Term Memory (BiLSTM) method.

RESEARCH METHODS
The data used in this study was obtained from the results of crawling on Twitter related to Permendikbud 30/2021 in Indonesia in November 2021. From the crawling results, 10,000 tweets were obtained. The data will be pre-processed before analysis and will then be categorized into two, namely pros and cons. The flowchart that is annexed to Figure 1 shows the research methodology that was used to carry out this study.

Text Mining
Text mining is an important data mining technique that includes the best ways to find patterns [7]. Text mining is a method of conducting knowledge searches that focuses on data in the form of documents or texts with the goal of extracting and identifying useful information. Raw data from the results of text mining activities must be prepared in advance according to the needs of the analysis so that it can produce good analytical results [8]. This is where initial processing or preprocessing of text data is needed.

Deep Learning
Deep learning can be considered the development of an artificial neural network. In deep learning, a system can learn to classify directly from images or voice [9]. Deep learning is very well suited to apply to supervised learning, unsupervised learning, and semi-supervised learning, as well as for reinforcement learning in various applications like the introduction of the image of, the sound of, classifications, text, and so on. Modeled on the DL, it is basically built on an artificial neural network (neural network). If a network has more than three layers, the network includes a deep network [10].

Synthetic Minority Oversampling Technique (SMOTE)
The synthetic minority oversampling technique (SMOTE) is one method that can overcome data imbalances. This method increases the number of datasets in the minor class by generating synthesis data so that the number of datasets for the minor class and major class are balanced. Synthetic data is created based on k-nearest neighbors. Synthesis data generation for numeric and categorical data is different. Numerical data is measured by its proximity to the Euclidean distance, whereas categorical data is measured by the mode value. The calculation of the distance between examples of minor classes whose variables are on a categorical scale is carried out using the Value Difference Metric (VDM) [11].

Bidirectional Long Short-Term Memory (BiLSTM)
Long Short-Term Memory (LSTM) is the development of the recurrent neural network (RNN) architecture to deal with the vanishing gradient problem [12], where the slope of the loss function decreases exponentially when processing long sequential data. This problem causes the RNN to fail to capture longterm dependencies so that it can reduce prediction performance [13]. LSTM replaces the RNN layer with a memory cell block using a gate mechanism consisting of a forget gate, an input gate, and an output gate. Similar to an RNN, a LSTM is composed of neurons that are processed repeatedly.  The forget gate is the first gate on the LSTM that accepts input ℎ −1 and to produce a value of 0 or 1 in −1 . When the forget gate is 1, the cell state will store information, whereas if the value is 0, the information will be removed from the cell state. The input gate is the second gate in the LSTM to determine what information will be stored in the cell state consisting of the sigmoid layer and the tanh layer. The sigmoid layer decides which value to update, and the tanh layer creates a new value to be added to the cell state. The output gate is the last gate on the LSTM and determines the output of the cell state [11]. The LSTM flow is arranged in the following equation [14].
where , , , is trained weights, , , , is trained biases, is function sigmoid, is input in time , ℎ −1 is hidden state previous time, is cell state in time , ℎ is output in time , is forget gate in time , is input gate in time , is output gate in time .
LSTM only reads sentences in one direction, from beginning to end [15]. Bidirectional long short-term memory (BiLSTM) is a development of the LSTM model where there are two layers whose processes are in opposite directions to each other. This model is very good for recognizing patterns in sentences because each word in a document is processed sequentially, because reviews can be understood if learning is sequential for every word. The layer below moves forward, that is, understands and processes from the first word to the last word, while the layer above moves backward, that is, understands and processes from the last word to the first word [6]. The two hidden states ℎ and ℎ from the LSTM are combined to form the final hidden state ℎ , as expressed below With this layer of two opposite directions, the model can understand and take perspective from the previous word, and the leading word, so that the learning process will be deeper, which will impact the model's ability to better understand the context in the review. Figure 3 is BiLSTM Architecture.

Preprocessing Data
The dataset in this study was obtained from the results of crawling social media related to Permendikbud 30/2021 in Indonesia in November 2021. 10,000 tweets were obtained from the crawling results. The data will be preprocessed before analysis. The data obtained from the results of the crawling process still contains capital letters, characters or punctuation marks, incorrect spelling, and words that are less meaningful, which can affect the analysis. Thus, a preprocessing process is needed to clean the data. There are several preprocessing stages. The first is cleaning, namely removing symbols and punctuation marks, numbers, URLs, hashtags, usernames, and RTs from tweet data. Case folding, which is changing the use of capital letters to lowercase letters or lower case on tweet data; and filtering, which is removing words that are less needed and do not convey a significant message such as pronouns, conjunctions, and others in a text based on a stopword dictionary. It is normalizing, namely normalizing non-standard words or words in foreign languages into words that are in accordance with the KBBI. The next stage is the tokenizing stage, which involves breaking sentences into words that do not affect each other, which are called tokens, and finally the stemming stage is the stage to change words to their basic form by removing affixes.
The data from the crawling results still contain duplicate or twin data caused by tweets that are posted repeatedly. So, it is necessary to delete duplicate data manually. After deleting duplicate records, 1926 records were obtained from 10,000 records. Furthermore, the Permendikbud data were manually labeled into two categories, namely "pro" and "con." This labeling uses the researcher's point of view. A tweet or tweet is included in the "pro" category if the tweet contains positive words, not badmouthing, and words of support for Permendikbud 30/2021. Meanwhile, a tweet or tweet falls into the "contra" category if the tweet contains negative words and tends to vilify Permendikbud 30/2021. Based on the labeling results, 1609 data points are in the pro category, and 317 are in the con category. Table 1 is the data that has been labeled. The datasets that have gone through text preprocessing and labeling are then separated by spaces using a tokenizer from the Keras library. Next, the list of vocabulary tokens is converted to a numerical sequence by replacing the index of each vocabulary with an integer value.

Balancing Data
The research data held are unbalanced data because the amount of data in the class is less or more than the number of other classes, which results in inequality in one class. In classification, imbalanced data can affect the performance of the analysis. One method to handle imbalanced data is SMOTE. The SMOTE method is to balance the amount of data by increasing the number of minority class data by generating random data so that it is balanced with the number of majority class data. Figure 4 is a data plot before data balancing with SMOTE is carried out.  Figure 4 shows a plot prior to SMOTE, with the number of data points labeled "pro" at 1609 and the number of data points labeled "con" at 317. It can be seen that the difference in the amount of data between the pro and con categories is quite significant. So, to avoid errors in classification, data balancing is carried out with SMOTE so that the amount of data is balanced. Figure 5 is the plot after SMOTE.  Figure 5 shows the plot after SMOTE. After balancing the data using SMOTE, the amount of data in the "con" category with label 1 increased from 317 to 1609 label 1 increased from 317 to 1609. It can be seen that the "con" category, which is the minority category, has balanced the amount of data on the majority class, namely "pro."

Splitting Dataset
After the data balancing process has been carried out, the next process is splitting the dataset into training data and testing data. Training data aims to train the algorithm. Testing data is data used to test the accuracy and performance of the previously trained model. In this study, we used a comparison of 80% for training data and 20% for test data.
The total data after data balancing is 3218. After sharing 80% of the training data, a total of 2574 training data sets are obtained, with 1287 in each category. Then, for 20% of the test data obtained, 644 were obtained, with 322 data for each category ("pros") and "cons" obtained.

Modelling Data Using Bidirectional LSTM
The next process is the classification of tweets that are included in the "pro" and "con" categories to get the best model using the bidirectional LSTM method. Before classification with bidirectional LSTM, it is necessary to first make a model with several parameters. The model made is a sequential type, which is a model made on a layer-by-layer basis. In addition, the researchers did hyperparameter tuning to obtain the best parameters from the bidirectional LSTM model in order to produce better performance. The GridSearchCV tuning method was used in this study. The best hyperparameter tuning parameters will be used to create a new model that will outperform the previous model in classification. Table 2 are the hyperparameters used in the tuning and the optimal parameters obtained. The classification process is done on the training data using the BiLSTM method with optimal parameters found through hyperparameter tuning. This is done to train the algorithm and study data patterns from the training data. Figure 6 is a plot of how well the classification worked when BiLSTM was used with training data. Based on Figure 6, it can be seen that the training accuracy increases and decreases with each increase in epoch. The highest accuracy is achieved when the number of epochs is 25. For the accuracy of the training data, the classification results have an accuracy of 88.51%.

Evaluation Model
After classifying the training data using optimal parameters, the classification process is carried out on the test data to see the performance and accuracy of the previously trained algorithm. the BiLSTM method. Table 3 is the result of the confusion matrix using the BiLSTM method. There are 306 correct data points in the pro category and 257 correct data points in the con category. From the results of the confusion matrix, calculations were made to obtain the values of accuracy, precision, and recall. a) Accuracy Accuracy is a value that indicates the level of closeness between the predicted value and the actual value.

CONCLUSIONS
Based on the results and discussions that have been explained, we have some conclusions. a). The data used is tweet data regarding Permendikbud 30/2021, which was taken from Twitter using a crawling technique. We obtained 10,000 records from the crawling results. After manually preprocessing and removing duplicate data, 1926 records were obtained. The information was divided into two categories: pros and cons. The data for the pro category is 1609, and the data for the contra category is 317. b). Classification analysis employs the bidirectional LSTM method, which compares 80% of training and 20% of test data, respectively. To get optimal parameters, hyperparameter tuning is used. The optimal parameters for classification are batch size: 128, dropout: 0.2, embedding: 64, epochs: 25, hidden unit: 16, learning rate: 0.01, and optimizers: Adam. These parameters yielded 87% accuracy, 82% precision, and 96% recall.