Antisocial Behavior Identification from Twitter Feeds Using Traditional Machine Learning Algorithms and Deep Learning.

Antisocial behavior (ASB) is one of the ten personality disorders included in ‘The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and falls in the same cluster as Borderline Personality Disorder, Histrionic Personality Disorder, and Narcissistic Personality Disorder. It is a prevalent pattern of disregard for and violation of the rights of others. Online antisocial behavior is a social problem and a public health threat. An act of ASB might be fun for a perpetrator; however, it can drive a victim into depression, self-confinement, low self-esteem, anxiety, anger, and suicidal ideation. Online platforms such as Twitter and Reddit can sometimes become breeding grounds for such behavior by allowing people suffering from ASB disorder to manifest their behavior online freely. In this paper, we propose a proactive approach based on natural language processing and deep learning that can enable online platforms to actively look for the signs of antisocial behavior and intervene before it gets out of control. By actively searching for such behavior, social media sites can prevent dire situations leading to someone committing suicide.


Introduction
Antisocial behavior is one of the ten personality disorders in 'The Diagnostic and Statistical Manual of Mental Disorders (DSM-5). These ten disorders are characterized into three clusters: antisocial personality disorder falls in Cluster B, Borderline Personality Disorder, Histrionic Personality Disorder, and Narcissistic Personality Disorder [1]. It is a prevalent pattern of disregard for, and violation of the rights of others. A person with antisocial personality disorder fails to conform to social norms concerning lawful behavior. The person can become irritable, aggressive, and consistently irresponsible when dealing with others. The person may lack remorse and mistreat others [1,2]. Many them from using these platforms [10]. Online antisocial behavior appears to be an Internet manifestation of everyday sadism. An individual who possesses and displays such behavior online seems to enjoy it at the expense of others ignoring the distress and harassment it may cause [11]. Apart from Sadism, attention-seeking, boredom, a desire to cause damage to the community, and revenge are some of the motivations related to antisocial behavior [12]. Antisocial behavior annoys and interferes with a person's ability to conduct business lawfully. Current measures to discourage antisocial behavior rely mainly on users reporting it directly to platforms [13]. In most cases, victims are reluctant to confront such behavior online because they fear retaliation. Therefore, most cases of antisocial behavior go unnoticed. Online platforms encourage freedom of speech but fail to distinguish between free speech and unacceptable behavior. Current measures do not effectively prevent people from explicitly displaying antisocial behavior, exposing many who may fall into a vulnerable group of people, to be on the receiving end of such behavior. Twitter is among the most popular social media platforms encouraging people to share views and content. A user contributes in the form of a tweet, which is 280-word text and may contain an image, a video, a link to an article, etc. The platform encourages user participation in discussions on topics of interest, however, this may bring along some undesirable behavior, such as bullying, abuse, and harassment [2]. Online antisocial behavior is prevalent mainly among users aged 18-27 and has also been linked to excessive use of online platforms. Perpetrator seems to enjoy at the cost of others [11,12,14]. Ramifications of excessive use also lead to other psychological disorders, and employing measures to curtail their impact on society is imperative [15].

Motivation
Twitter and other platforms rely on users to report Antisocial behavior to the media. Only based on the user initiative, Twitter intervenes and looks into matters of various things. The platform has automated systems that may prevent the distribution of illegal material, spam, nudity, pornography, etc. but nothing for antisocial behavior [16]. So, on the one hand, the platform connects users to enable the exchange of information, ideas, and other valuable resources. On the other hand, however, it facilitates the spread of antisocial behavior and related issues and puts many people at risk [17]. This paper studies the problem of antisocial behavior on public platforms.

Our Approach
We in this paper propose an approach based on natural language processing (NLP) and deep learning that can be used, on a scale, to classify tweets containing antisocial behavior with high accuracy and precision. The approach can be implemented to automate the detection of antisocial behavior on Twitter and other online platforms to curb its prevalence. To our knowledge, no prior work has either focused on critical tweet identification or evaluated Deep Learning, and machine learning techniques against different feature extraction approaches for identifying psychological disorders, particularly antisocial behavior, from social media data. Firstly, a benchmark data set of tweets labeled "ASB" and "non-ASB" is constructed. Textual features are then extracted from unstructured data for deep learning, a cutting-edge machine learning architecture, along with traditional machine learning algorithms, to build a prediction model for the automatic identification of ASB. We have treated the problem of recognizing ASB online as a binary text classification task, where a tweet is classified as "ASB" or "non-ASB" based on the textual content. Table-1 shows examples of antisocial and non-antisocial tweets classified by our approach.

Contribution
The main contributions of this research study are: (1) A medium-scaled benchmark antisocial behavior tweet dataset was constructed with labels for antisocial and nonantisocial behavior tweets. (2) We developed a deep learning classification model after evaluating the performance of different DL architectures. (3) We validated the higher performance of Deep Learning models against several traditional machine learning models. (4) We also presented a visually enhanced interpretation of different feature vectors in machine learning (5) Proposed a novel approach to study psychological disorders from social media data using artificial intelligence The rest of the paper is organized as follows. Section II provides the background on antisocial behavior disorder, etiology, manifestation, and repercussions. Section III presents an approach to ASB tweet identification. Section IV offers details on experiments conducted to evaluate our system with analyses of the results and discussion. Section V concludes our work in this paper and envisages future research directions.

Online Antisocial Behavior
To understand antisocial behavior online, we went through the diagnostic criteria explained in DSM-5 [1]. The term antisocial personality disorder is primarily used in a clinical setting and may be used to describe the behavior of a person against societal norms. To be antisocial may mean to be against rules, laws, standards, and acceptable behavior [2]. Furthermore, against the government and regulation may refer to a failure to obey laws and legal system, engaging in criminal activities, arrest, etc. A person with an antisocial personality may lie, deceive, and manipulate others for self-amusement and profit. The person may get irritable and aggressive quickly and is inclined to engage in fights. The person may also be impulsive, irresponsible, and lack remorse for action [8]. A person's writing can diagnose not all psychological disorders, but a few can and antisocial behavior is one of them. Since it can be interpreted by how a person writes, we can detect such behavior online from tweets, posts, reviews, and comments. In any text, antisocial behavior is expressed by using words and the context of the use of those words. There are several rude and taboo words and short phrases that can be associated with antisocial behavior. It may seem easy for a human to pick up such behavior through text; however, it may not be that easy for a machine [18]. One reason is that some rude words can be used in humor or sarcasm, which may not always be considered antisocial. Also, the context of a text plays an important role in classifying it as an antisocial text. The use of slang, the order of words, local culture, etc., all play an important role in classifying a text. Some words and phrases that are normal to use in one country may imply rude or antisocial behavior in other. An example is an experience shared by a friend from Australia, who was in a café in the US and asked for a 'White Coffee". This is a standard way of getting a coffee with milk in Australia; however, in the US the guy at the café, who was a person of color, thought that my friend was rude and racist. My friend should have asked for 'coffee with milk' instead of 'white coffee.' Under certain circumstances, it is difficult for even humans to know the exact intentions of a person from their writings, and we can imagine how hard this could be for a machine. A machine or a computer relies on a set of rules and instructions to take any action; however, in the case of natural language processing, it could be more straightforward. There are few techniques used in natural language processing, and we, in this research project, have used a machine learning approach. To train a machine learning model to detect antisocial behavior from a person's writing requires a lot of training and testing data, along with ground truth validation. We sought the help of a psychology graduate to label our dataset and ground truth validations.

Etiology of Antisocial Behavior
Understanding the etiology of antisocial behavior may be the first step toward preventing and eliminating it. Antisocial behavior disorder is part of Cluster B of personality disorders and borderline, histrionic, and narcissistic personality disorders. Individuals who have these behavioral personality disorders appear emotional, dramatic and erratic. These characteristics are common in all these four disorders in the cluster. A person with antisocial behavior often disregards other people's emotions and feelings and often engages in activities that are considered illegal; however, the manifestation of such activities dwindles as the person grows older [1]. There may be many elements that lead to a person developing antisocial behavior. Some of these are genetic influences, maternal depression, Parental-rejection, physical neglect, poor nutrition intake, and adverse-socioeconomic and sociocultural factors. [1,[3][4][5][6][7]19] These factors can be categorized broadly into three main categories: Neural, Genetic, and Environmental [8,20,21]. Antisocial behavior due to neural factors has been studied through structural and functional approaches. Structural studies assess the brain's morphology, and available studies assess its activity. Together these studies try to understand core neural regions that are related to salience detection, affect, and controlled cognition, including the frontal cortex, amygdala, and anterior cingulate cortex [8] A person's genes are linked to the antisocial behavior he may develop during adolescence [3,[22][23][24]. Certain types of gene combinations are closely associated with such behavior. A child raised by biological parents diagnosed with antisocial behavior is highly likely to develop antisocial behavior. However, some studies have concluded that if the same child with parents diagnosed with antisocial behavior is raised by adopted parents, who do not suffer from the such disorder, he has a lower chance of developing antisocial behavior [20,25,26]. Therefore, genes play an essential role in the onset of antisocial behavior in a person. However, the impact can be mitigated if the individual's environment can be more positive. Some environmental factors that may trigger or lead to antisocial behavior are exposure to community violence, family dysfunction, and peer influence [8,27,28].
Research has shown that being part of a disadvantaged community, living in a poor neighborhood, dependent on social security, being a part of female-headed households, and not having a job may exaggerate or trigger the onset of antisocial behavior [29][30][31][32]. Being a part of a broken family and facing maltreatment by either parent, a parent's mental health can also impact an individual's mental health. Apart from parents, abuse by others around an individual can also prompt him to manifest antisocial behavior [33,34]. Child neglect, in general, has been associated with ASB [35]. The sort of company an individual hangs around with usually influences his behavior and personality, and vice-versa. So hanging around with an individual who manifests antisocial behavior can lead you to display such behavior as well [36][37][38][39][40][41]. We all know that smoking harms the person who smokes and individuals around him who inadvertently and passively inhale smoke excreted by a smoker. Many studies have linked maternal smoking during pregnancy and severe mental disorders, particularly antisocial behavior disorders in offspring [4,[42][43][44][45][46][47][48][49]. Similarly to smoking, excessive parental drinking is also associated with an offspring developing ASB. Apart from neural, genetic, and environmental factors, some studies have found a link between poor-quality nutrition and childhood antisocial behavior [7]. A deficiency of B-Vitamin is mainly linked with ASB and other mental health & behavioral disorders [50]. So, many factors can lead to an individual developing general mental health disorders and antisocial behavior. Here, we have discussed some of the crucial factors leading to ASB, and since this is still an active area of research, we may learn more about this personality disorder in the future.

Manifestation of ASB
Antisocial behavior emerges in disparate forms online. Some of the most common ones are trolling, cyberbullying, threatening, hostile behavior, offensive language, the publication of inappropriate images, etc. Trolling is widespread on social media, magazines, and news websites. Trolls are general visitors to a website and write offensive and inflammatory comments in the public section. Their main aim is to disrupt an online discussion and, at the same time, grab some attention in the process. They disregard the author of the writing they comment on and show no respect to other commenters. They do this by posting comments that are sexist, hateful, racist, and profane in nature. Troll intensity range from subtly provoking someone to outright threatening and abusing [51]. For some, trolling traits are inborn, and they have a history of trolling and engaging in such behavior online. These people seem to enjoy at the cost of others [11,12,14]. This type of trolling is associated with sadism [52]. For others, environmental variables, situations, and context can come into play [53]. Negative mood and seeing other people trolling online can also thrust someone into trolling [54]. A person who otherwise has a charming and normal personality can sometimes be pushed into getting involved in trolling inadvertently. This sort of situation may arise if someone, who is not a troll, feels that he has been pushed around and needs to stand up to it. In the process, the victim himself can start trolling the abuser, either to get him off his back or to teach him a lesson, in the hope of preventing him from engaging in trolling in the future [20,54]. Studies have found that people who troll focus their effort on a small number of threads and make issues of petty things. They usually write worse than people who do not troll, and, in some instances, their writings are irrelevant to the topic of discussion. Considering gender from a perspective, males are more likely to get involved in trolling compared to females [16,17,55]. Over time, these trolls become less tolerant of the online community and get reported and kicked out of the conversation and, in some cases, from the community [2,56]. The impact of trolling on victims can sometimes be more devastating than if they have experienced similar behavior in real life [57]. Exposure to online trolling can lead the victims to experience psychopathological outcomes such as anxiety, depression, and low self-esteem [58]. While trolls mainly focus on being a nuisance and attracting attention, cyberbullies target individuals. Instead of posting general offensive and inflammatory statements in the public comment section of a website, they post abusive and vicious comments about a single individual. Cyberbullying refers to using an online platform, such as Twitter, Reddit, Facebook, etc., to intentionally and repeatedly harass or harm an individual [59]. Cyberbullies focus on intimidating, shaming, and demeaning their victims. Unlike trolls, cyberbullying does not usually want to attract attention and instead focuses more on targeting an individual and causing distress to them. For this, they post images, text, audio, and video targeted at individuals repetitively. [59] This media is abusive, aggressive, and intentionally drafted to bully someone online. As the use of the online platform has increased, so has cyberbullying, which is prevalent in school-aged children. Depending on the measuring tool applied, 10%-40% of school-age children experience some sort of cyberbullying [60]. Cyberbullying has recently been getting much attention from government authorities and social scientists because of its association with many suicides [61]. Trolling and cyberbullying are the two most prominent manifestations of antisocial behavior online. Threats, misleading and wrong information, offensive language, sexism, racism, and rude and taboo words are other ways antisocial behavior can take form online.

Repercussions of ASB
Antisocial behavior and its impacts are matured and wellresearched; however, online antisocial behavior is a relatively new research area and has recently gained plenty of attention. The perpetrator often displays such behavior via cyberbullying and trolling. Individuals on the receiving end of antisocial behavior get impacted in many different ways, and adverse health impact is one of them. Victims can suffer internalizing problems such as depression, low self-esteem, anxiety, suicidal ideation, and anger [62]. They can also go through externalizing such problems as alcohol abuse, smoking, self-harm, aggression, and hostile behavior towards the external environment [63,64]. Victims of all ages can experience negative mental health impacts of antisocial behavior; however, individuals who come across such behavior as children have higher chances of developing the psychological disorder and social problems [65][66][67]. Some studies have linked exposure to antisocial behavior as a child, a drop in academic performance as a young child, and low-income family and social relationships as an adult [68,69]. Victims are often preoccupied with their antisocial behavior experiences and find it hard to concentrate on academic tasks, leading to poor performance. In addition, to drop-in grades at school, a drop-in attendance has also been linked to exposure to such behavior, leading to a vicious cycle affecting all aspects of academic life. Adult victims of antisocial behavior report a higher level of anxiety, depression, and severe social difficulties. The thought of experiencing such behavior repeatedly prevents many victims from going out and socializing. This leads to self-confinement & isolation, leading to depression and social problems [70]. Antisocial behavior at work leads to lower employee morale and lost output. Perpetrators often target their victims via inflammatory emails, offensive text messages, and posting inappropriate comments and images. Females, minorities, and new employees are often easy targets for such behavior. Putting measures in place and managing such activities & behavior at the workplace cost organizations many resources, in addition to negative media coverage and higher staff turnover, which again adds to the cost of doing business [71][72][73][74][75]. Many studies have linked victims of antisocial behavior to drug and alcohol use, hyperactivity, and a decline in prosocial behavior [76,77]. Victims fall prey to drugs and alcohol as a convenient escape to their problems, and excessive use makes them hyperactive and deters them from socializing. Studies have also linked suicidal thoughts and self-harm behavior as one of the ramifications of experiencing antisocial behavior. Self-harm may include cutting, jumping from heights, self-battery, burning, and poisoning, with some industrialized and developed nations experiencing higher-than-average incidents [78][79][80]. In contrast, some victims may use aggression to get their frustration out and may bully, harass or troll other individuals around them [81]. Despite its relatively brief history, online antisocial behavior has been identified as a severe public health threat. Apart from the direct impact on victims and an indirect impact on their family & friends, antisocial behavior is also a burden on the public health system. The cost of treating individuals with depression, anxiety, and other related psychological disorders adds up and significantly impacts the growing public health spending [71,82].

Obligation to Restraint
Online antisocial behavior needs to be deterred and confined. We may not be able to eliminate it; however, by placing appropriate measures in place, we may be able to confine it to a certain extent or reduce its impact on victims, their families, the workplace, and the public healthcare system. Antisocial behavior is a massive cost to our society. With the advent of the internet, it has become ever easier for many people to indulge in such behavior online. The spike in the incidents of antisocial behavior in general and online antisocial behavior, in particular, can be explained by the fact that perpetrators can stay anonymous online, which usually is not an option for them in a face-toface situation [83]. In the real world, perpetrators typically have power over victims they exploit to bully and harass. This power could come in many forms, such as social status, physical strength, and workplace seniority. In an online world, these powers may be insignificant. Since a perpetrator can stay anonymous, he can also bully and harass someone higher in a hierarchy regarding workplace seniority, status, and physical strength [84]. Also, in a faceto-face confrontation, a perpetrator can seize abuse and bullying once he recognizes that he has hurt his victim enough. In online bullying, a perpetrator may not know when to stop and can push the victims toward extreme actions such as self-harm or suicide. Pushing someone online to the extent that may lead the victim to commit suicide is a form of suicide baiting, using which an offender encourages a victim to take their life [85]. Online antisocial behavior is a huge cost and burden to our society. Damages caused by this can be seen in families, workplaces, and the public healthcare system. This kind of behavior is never accepted, whether online or offline. Even though large social media platforms and other online platforms are responsible for ensuring their platforms do not become breeding grounds for antisocial semantics, it is everyone's responsibility, who use those platforms to discourage and report such behavior. Our proposed approach in this paper can assist platforms to automatically identify such behavior on a scale and restrain it from spreading.

Natural Language Processing
Natural language processing is a field concerned with the ability of a computer to understand, analyze, manipulate, and potentially generate human language. By human language, we refer to any language used for everyday communication. This can be in English, Spanish, French, or Mandarin. A programming language such as Python, which we have used in this research, does not naturally know what any given the word means. All it sees is a string of characters. For example, it has no idea what antisocial means. It seems it is a ten-character long word, but the individual character doesn't mean anything to Python.
Indeed, the collection of those characters together does not mean anything either. Humans know what an 'A' and a 'S' means, and together, those 10 characters make up the word 'antisocial', and we know what that means. So natural language processing is the field of getting the computer to understand what 'antisocial' signifies, and from there we can get into the manipulation or potentially even generation of that human language [74,86]. We probably experience natural language processing daily without even knowing. Natural language processing is a broad and evolving field that encompasses many topics and techniques. The core component of natural language processing is extracting all the information from a text block relevant to a computer understanding of the language. There are many techniques for natural language processing and machine learning methods in general and deep learning, in particular, is the most promising of all. Machine learning is a field of study that allows computers to learn without being explicitly programmed.

Methodology
This section presents our approach based on natural language processing and machine learning that can automatically detect antisocial behavior online and can enable platforms such as Twitter to proactively prevent it from spreading by having appropriate measures in place. Most of the research on antisocial behavior has been qualitative, focusing mainly on deep case study analysis. Study groups are often chosen manually and are small in number. These studies could be more convenient and may require a lot of resources and time. In today's world, we humans spend most of our time online. Access to the Internet has changed how we live and do things in our daily lives. We spend more time in front of screens today than we ever did. Most of our daily tasks, such as work, social interactions, banking, shopping, entertainment, etc. occur online. Since the way we live and do things have changed significantly, we need new ways to explore and study personality and behavioral traits [17,34]. The research for this project has been conducted by collecting data from the social media site Twitter. Since this data is generated during our interactions with the outer world, it has much information about our human behavior and personalities. In this research project, we extract such information and use it to build machine learning and deep learning models to detect antisocial behavior online. The proposed approach consists of collecting the right tweets and labeling them. Once marked, these tweets and labels need to be verified by a qualified person. The qualified person in our scenario is a person who has a thorough understanding of psychological disorders and can diagnose them in a clinical setting. Once the data, a set of tweets in our case, is properly labeled, we can use natural language processing techniques to clean and pre-process it. These natural language processing techniques are discussed in detail in the following sections.

Data Extraction
We collected 55,810 tweets from Twitter between Oct 2018 and Feb 2019. After removing retweets and duplicates, we were left with 25,500 tweets. Tweets are 280-word text that users share with others on the Twitter platform. When Twitter first started, the limit on the length of these tweets was 140 words and was later increased to 280 words as the platform's popularity soared. We used various phrases such as "I do not care about the law," "I wish you die soon", "Go to hell" etc. to search and collect these tweets. Text data collected online is typically in a semi-structured or unstructured form. Our data was no different and was in semi-structured from when first collected. Therefore, some tweets were missing delimiters and did not indicate any punctuation. We used functions from the NLTK library of Python to structure our data set. Once the dataset was in a structured form, we annotated the dataset manually with two categories: Tweets that conveyed antisocial behavior and tweets that did not. Once the dataset was annotated, we wanted to get it verified by someone from Psychology. We hired a psychology graduate to do so. The person thoroughly understood all the personality disorders and could diagnose them in a clinical setting. Psychological disorders can be classified into personality, behavior, and State of mind. Behavior and State of mind disorders fluctuate and usually cannot be detected in a person's writing. Behavior and State of mind may change from time to time. However, Personality disorders or traits do not fluctuate and stays with a person for a longer period [1,87]. Since these traits stay with a person longer, they manifest through a person's speech and online writings. Antisocial behavior is a personality disorder that can be reliably detected from online corpora. Our annotator could manually review tweets to see if they qualified as ASB tweets. If a tweet did, it was labeled one. Once labeled, our dataset was ready to be explored further.

Data Pre-processing
This phase involved removing punctuation from our tweets, followed by tokenization, which means dividing a sentence into individual words. Once tokenization was done, the next step was to remove stop words. Stop words are words that do not contribute much to the meaning of a sentence. Examples of such words are the, is, are, etc. After removing the stop words, we used stemming to cut down words into their shortest form. This is done to reduce the work for our algorithm. All these steps are explained in the following paragraphs. The first step in the pre-processing phase was to remove punctuation from the tweets. To remove punctuation, we had to show Python, the programming language we used, what punctuation even looked like. We accomplished by using the String package in Python. We care about removing punctuation because period, parentheses, and other punctuations look like just another character to Python. Still, realistically, the period does not help pull the meaning out of a sentence. For instance, for us "I like to research." with a period, is the same as, "I like to research". They mean the same thing for us, but when we give these sentences to a machine learning algorithm, the algorithm says those are not equivalent. We wrote a function to cycle through every character, checked if it was punctuation, and discarded it if it was. This was done to reduce the workload of our algorithm. By removing punctuations, our algorithm had to deal with fewer characters in the learning process. Now that we had removed punctuation, we could begin tokenizing our text. Tokenization is spitting some string or sentence into a list of words by white spaces and special characters. For example, we could split the sentence "I am doing research" into four words: 'I', 'am', 'doing', and 'Research'. Instead of seeing the whole sentences, our algorithm could see four distinct tokens, and it knew what to look at. Some of the words were more important than others. For instance, the words 'the', 'and', 'of', and 'or', appear frequently but offered little information about the sentence itself. These are what we call stop words. We removed these words to allow our algorithms to focus on the most key words in our tweets. From the example above, if we remove 'I' and 'am', we are left with 'doing research'. This still gets the most important point of the sentence, but now our algorithm is looking at half the number of tokens. The next step in the process was stemming. Stemming is reducing inflected or derived words to their word stem or root. In other words, it means to chop off the end of a term, to leave only the base. This means taking words with various suffixes and condensing them under the same root word. For example, we can stem words such as 'connection', 'connected,' and 'connective' to one word 'connect.' Stemming shoots for the same goal by reducing variations of the same root word and making our algorithm deal with fewer words. Without stemming, our algorithm must keep all three words: 'connection,' 'connected', and 'connective' in memory, increasing the workload and making our machine learning model less efficient. In simple words, the whole idea of all these steps is to reduce the corpus size for our machine-learning model to deal with. For stemming, we used the Porter stemmer from the NLTK package. The deep Learning algorithms we implemented in this paper do not require similar pre-processing as traditional machine learning algorithms. We still had to remove duplicates and retweets and had to structure the data set to iron out any abnormalities and website links.

Model Construction
The natural language toolkit is the most utilized package for handling natural language processing tasks in Python. Usually called NLTK for short, it is a suite of open-source tools initially created in 2001 at the University of Pennsylvania to make natural language processing in Python easier. NLTK is great because it provides a jumpstart to building any natural language processing tasks by providing essential tools that can be chained together rather than making them from scratch. We used the NLTK package of Python for the traditional machine learning algorithms.
Once we had clean text data that we could use to build our machine-learning model, we needed to convert it into a form that our model could easily understand. The process is called vectorization. This is the process of encoding text as numbers to create feature vectors. A feature vector is an n-dimensional vector of numerical features representing some object. In our context, we had to convert individual tweets into a numeric vector representing those tweets. We did this by taking our dataset, which had one line per document, with the cell entry as the actual text message, and converting it into a matrix that still had one line per document. Still, then we had every word used across all documents as the columns of our matrix. And then, within each cell was counting, representing how many times that particular word appeared in that document. This is called a document-term matrix. Once we had the numeric representation of each tweet, we carried it down with our machine-learning pipeline and fitted and trained our machine-learning model. We vectorized text to create a matrix that only had numeric entries that the computer could understand-in our case, counting how many times each word appeared in each tweet. A machine-learning model understands these counts. If it sees a one, a two, or a three in a cell, the model can start correlating that with whatever we're trying to predict. In our case, that was antisocial behavior. We analyzed how frequently certain words appeared in a tweet in context to other words to determine whether the tweet manifested antisocial behavior. We used both Word Frequency (WF) and Term Frequency-Inverse Document Frequency (TF-IDF) vectorization methods in this research. We did this to see a difference in the results with our machine learning model. The count vectorization created the document-term matrix and then counted the number of times each word appeared in that given document, or tweet in our case, and that is what is stored in the given cell. The equation for this is: Term Frequency-Inverse Document Frequency, which is often referred to as TF-IDF, created a document-term matrix, where there was still one row per tweet, and the column still represented a single unique term; however, instead of the cells representing the count, the cell represents a weighting that was meant to identify how important a word was to an individual tweet. We started with the TF term, which is the number of times a term occurred in a tweet divided by the number of all terms in the tweet. For example, if we use "I like research" and the word we focus on is 'research' then this term would be 1 divided by 3 or 0.33. The second part of this equation measures how frequently this word occurs across all the tweets. We started by calculating the number of tweets in the dataset and divided that by the number of text messages that this word appeared in and then took the log of that equation. For example, if we had 20 tweets and only one had the word 'research', then the inverse document frequency means log (20/1). We had two parts of the equation: Term and inverse data frequency. The last step was to multiply both to get a weight for the word 'research' in the tweet. The equation is as follows: (2) Both the matrices have the same shape; the only difference is the values in the cells. After vectorization, we had our data set that algorithms could use to build a machine learning model. Machine learning is the field of study that allows computers to learn without explicitly programming. We do that by training a model using data and an algorithm and then testing its accuracy using more data. To this end, we divided our dataset into buckets to train and validate our model. In this project, we used the K-fold Cross Validation method to divide our data, using tenfold Cross-Validation. The full data set was divided into ten subsets and the holdout's procedure was repeated 10 times. Each time, nine subsets were used to train the model and the tenth subset for testing it. Results were stored in an array, and the method was repeated 10 times with different testing sets each time. In the end, the average of all test results was taken to produce the final result. While building our model, we tried five traditional machine learning algorithms: Logistic Regression, Support Vector Machine, Random Forest, Decision Tree, and Naïve Bayes. These algorithms were implemented twice using two different vectorization methods: Word Frequency and TF-IDF. As discussed earlier, not much research has been conducted to deter online antisocial behavior, and therefore we felt the need to explore all the available machine learning algorithms to get an optimum result with our dataset.
For deep learning algorithms, we used Keras [88], an open-source neural network library in Python. We experimented with the four most popular algorithms for text analysis and classification: CNN, Bidirectional RNN, and Bidirectional LSTM, and GRU. Like the traditional algorithm we used ten-fold cross-validation to train and evaluate our deep learning models. We used word2vec feature extraction for all the four models

Performance Evaluation
The last stage in our model construction was to evaluate our proposed approach to detect and identify antisocial behavior post from social media. We adopted Accuracy, Precision, F-Measure, and Recall as the evaluation metrics for our algorithms. These metrics are widely used to evaluate performance for both machine learning and deep learning classifiers Field [83][84][85], and are appropriate for classifying posts and tweets related to antisocial behavior. We constructed one data set for the identification of antisocial behavior posts and non-antisocial behavior post. Therefore, adopting k-fold cross validation approach was imperative and was used. In this approach the collected dataset was arbitrarily apportioned into k partitions. Out of the k partitions, one was reserved as test subset and the others were combined into training subsets. The whole procedure was carried out k times, which was 10-times in our scenario. The results from all these 10 folds were then averaged to indicate an overall algorithm performance.

Prediction Performance Evaluation with Traditional Machine Learning
Online antisocial behavior is a relatively new area of research. When social media platforms such as Twitter and Facebook started getting traction, they bought in some of the issues along with them. Antisocial behavior is one of them. To the best to our knowledge, there hasn't been much work done to detect and prevent antisocial behavior online. There are studies on cyberbullying and trolling, which can fall under the umbrella term of anti-social; however, not much has been researched on detecting other aspects of such behavior. Using natural language processing and machine learning techniques, we have done a reasonably good job detecting all forms of antisocial behavior. Following are the results from trying five traditional classifiers and using count vectorization. The accuracy we got was relatively high with all the classifiers used. Precision, Recall, and F1 scores were similar with all these algorithms. All five algorithms detected antisocial behavior with high accuracy and precision. A tweet that was classified as containing antisocial semantics was the one that contained some sort of swear and rude word to upset or annoy someone. Not all the tweets that were classified positive contained swear words. The sentiment, semantic, and context of the text were also considered while manually labeling and deciding whether the tweet represented antisocial behavior. While classifying, some of the tweets were on the borderline or represented more of sarcasm than antisocial behavior. Such tweets were eliminated and were not used. Since this is one of the first studies trying to detect online antisocial behavior in all its forms, we wanted to keep the things simple for our algorithms and model. The tweets on which we had doubts to whether to classify them as positive or negative, were eliminated from the training and testing dataset. Since Most of our tweets were quite clearly positive or negative, it made the job of classifying algorithms much easier, as there was limited number of words and phrases that our model had to learn to distinguish between positive and negative tweets. This study can be further extended by adding more complex text to classify, even by some human standards. We assume that adding those sorts of tweets will impact the accuracy and precision metrics, however, it will enable our model to generalize better on any data set.
As mentioned above, we tried our classifiers with TF-IDF vectorization as well. The results are shown below. Support Vector Machine showed the best result when used with count vectorization; however, Random Forest was better when TF-IDF vectorization method was used. Overall, we managed to get good results with both vectorization techniques and were able to detect antisocial behavior from Twitter with high accuracy. The following charts show the similarities between accuracy, precision, recall and F1 score using two different vectorization techniques: Word Frequency and TF-IDF. In both cases, the results are highly similar. Reasons for such similar results could be the size of the dataset and the preprocessing techniques that we used. In regard to the size of the data set, even though we had around 55,000 tweets, more tweets could have bought in more variations in the text data. In regard to the pre-processing techniques, we believe our stemmer did a good job truncating all the important words to their roots, assisting both the vectorizing techniques to perform well. As can be seen from the charts below, Support Vector Machine and Logistic Regression performed the best and Naïve Bayes lagged behind in almost every measuring metrics. We propose the use of Support Vector Machine for our model based on its performance on our dataset and its overall credibility dealing with different types of datasets.

Performance Evaluation with Deep Learning
This section describes the four deep learning models used to conduct the experiment and the results obtained by using these models. First, we would like to tell the inner working of these models.
• CNN: The first model we tried was CNN and its detailed architecture is demonstrated in [89]. When a pre-processed tweet is fed into a Convolutional neural network, it learns the embedding or the text region internally and captures the semantic coherence information of the tweet. The first layer of CNN is known as the embedding layer and it extracts the ngram features and stores the word embedding for each word in the text. The convolutional layer contains a disparate number of computational units, each representing an n-gram from the text. Different combinations of n-grams can be experimented with, such as unigram, 2-gram, and 3-gram. The convolutional layers are of variable sizes and the pooling layer transmutes the previous convolutional representation to a higher abstraction level and outputs a fixed-size output. Lastly, a dense layer utilizes the combination of a product feature vector to predict a tweet.

•
RNN: The next model that we tried was RNN, and its architecture is described in [90]. RNN handles a flexible-length sequence input and has loops known as the recurrent hidden state. This loop apprehends information from earlier forms. At every step, it receives an input, which is used to update the hidden state. One benefit of RNN over CNN is that its hidden state integrates and utilizes information from previous time stamps. • GRU & LSTM: The last two models we experimented with were Bidirectional GRU [91] and Bidirectional LSTM [92]. Both GRU and LSTM are the improved versions of RNNs. They have memory units, that maintain and store historical information, and the gating units that regulate the flow of that information.
There is a subtle difference in the architecture of both.
LSTMs have three such gates, whereas GRU's have two. We experimented with the advanced version of GRU and LSTM, and these are called Bidirectional GRU and Bidirectional LSTM. Bidirectional features enable these architectures to store both future and historical information. Bidirectional features make GRU and LSTM state-of-art semantic composition machine learning architectures for text classifications. Applications in various fields can be found in [87][88][89].
For our study, we tried the above four mentioned deep learning architectures. We used 10-fold cross-validation, described in the previous section, to train and evaluate all these models. The results of all these models are shown in the following diagram. We can see the detailed performances in every iteration of the 10-fold crossvalidation technique for all four models, and the average of those iterations is also shown. The above figure compares the accuracy, precision, recall, and F1 scores. It also shows the Epoch, the number of cycles the algorithm went through to learn from the training set. The lower epoch may represent the undertrained model, and higher epoch indicates overfitting. Epoch between10-25 is considered a good outcome. We can see that the average epoch for our study for all these models lies between 9.6 -13.1. This is an indication that our models learned early on utilizing feature vectors. The following table presents the same results in a more compressed form, however, it shows only the averages, instead of every fold, for all four models used. It can be seen that the accuracy and precision of all these models are close to 100%.

Traditional Machine Learning and Deep Learning Comparison
In this study, we used both traditional machine learning and deep learning models. Both performed well on our data set of tweets; however, deep learning outperformed traditional methods marginally. The explanation for this outperformance is that deep learning models, unlike most traditional machine learning algorithms, can learn a text's semantics. As explained in the earlier section, these models have memory units and gates that can store and relay such information between different layers of architecture. These units and gates can enable complex data to be stored and communicated within the network, making it possible to handle even the large-scale information and assisting in learning. Learning features such as WC/TF and TI-IDF used in traditional methods cannot store and pass information, and they rely mainly on words and the number of occurrences of these words. SVM was the best performer from the traditional algorithms, and from the deep learning algorithms, CNN outperformed all the other algorithms. So, to build a model based on natural language processing and machine learning techniques, we propose Convolutional Neural Network architecture to classify tweets automatically on a large scale.

Semantic Coherence Analysis
This sub-section examines the data set to identify essential words in ASB and non-ASB tweets. ASB tweets contain mostly rude, forbidden, and taboo words, representing negative semantics and sentiments. Words such as F**K, mother**k, crime, smoke, lawless, screaming, bitch, fight, nigga, and enforcement are the most prevalent. These are not polite words and are usually avoided in social settings.
One will only use such words during a daily conversation if the intention is to offend and to manifest ASB.
On the other hand, non-ASB tweets are filled with encouraging words such as respect, others, like, beliefs, help, grateful, religion, respected, etc. The contrast in the use of words in both classes can be seen from the word cloud in figure 2. The words in large font are the ones that are prominent in both categories. There is a clear distinction between the type of words associated with ASB and non-ASB tweets. Some words appear more often in antisocial tweets than in nonantisocial tweets, and other words predominantly do not appear in these antisocial tweets. Traditional machine learning algorithms rely mainly on the meaning of words and how often these words appear in a text, and TF and TF-IDF feature extraction techniques depend on words' meaning. So even though these techniques learn to separate ASB and non-ASB tweets in the experiments we performed, they still need to capture the semantic relationship between these words fully. Deep learning, on the contrary, addresses this issue using the word embedding feature vector technique. In this technique, each word is represented by a feature vector that captures the semantics of a piece of text. Because of this ability of deep learning models, they can learn better from the training data and hence can perform better when implemented.
We also analyzed these words' support in identifying their corresponding class and their likelihood of occurrence in these classes. We picked up common, occurring words from both categories and calculated their support toward identifying tweets in that class. Some of the top words that occurred more than others are shown in the table-5 along with their percentage of occurrence and Z-scores. We performed a Z-test with a p-value <=.05 to establish statistical significance in the event difference.
We established from the experiments that taboo words such as shit, bitch, f**k occurred more often in antisocial tweets than non-anti-social tweets. People who exhibit antisocial behavior online often use taboo words. Some examples of tweets are: (1)  Words presented in Table-6 have exhibited some features in distinguishing antisocial and non-antisocial tweets. Solely relying on the term frequency may not be a very effective way of classifying these tweets automatically. The justification is that some taboo, bad, and threatening words are often used in non-antisocial tweets to spread awareness or report somebody to authorities. A good classification model should consider the semantic relationship of words in a piece of text rather than relying solely on word count, the approach common in traditional machine learning algorithms using TF-IDF and the Bag of Word approach. We in our experiments, implemented word embedding features in deep learning, in which a vector feature of 300 dimensions represented each word. Words with similar meanings usually have similar feature vector forms. These 300 dimensions captured the semantics of the tweets and the words used in them. A term used in two scenarios may represent a different meaning if the contexts of these scenarios are disparate. Furthermore, vector features of other but similar words may appear alike and display a strong correlation. To help understand this concept better, we have presented a visualization of the correlation between some of the ordinary occurring words in our dataset.
In figure 3, we have an identical set of words on both the x-axis and y-axis. The 289 small-colored squares represent correlations of words with other words in the figure. The diagonal from the top left to bottom right, made up of dark brown squares, shows the correlation of a term with itself. The darker the color, the stronger the correlation; on the contrary, the lighter the color weakens the correlation between words. As can be seen from the image, the word 'happy' has a high correlation with the words 'amazing,' 'grateful,' and 'thanks.' Similarly, 'asshole' correlates highly with the words 'bitch', 'fuck', and 'shit.' The expression 'behavior' is correlated to 'attitude,' and 'love' is related to 'happy,' 'grateful,' and' thanks.' The white spots within the figure show the opposite. Words 'happy,' 'amazing,' 'grateful,' and 'thanks' have no correlation with 'bitchass,' which shows that these words fall in the opposite semantic bucket to the term 'bitchass.' Lightcolor squares show no or weak correlation between words. The word embedding features of deep learning models can capture not just the actual meaning of words in a piece of text but also the context in which these words are used, enabling these models to perform better when compared with traditional machine learning models. The point to note here is that about our tweet dataset, the performance of deep learning models is slightly better than the conventional models because the models had to deal with fewer words due to the limited size of tweets, which is a maximum of 280-words, however, when we compare the same models on more extensive texts such as paragraphs or even a large documents, the difference between the performance widens, and deep learning models perform way better than the traditional models.

Conclusion & Future Work
This research introduces a data-driven approach to detect and prevent antisocial behavior online. Twitter is responsible for controlling its platform from becoming a breeding ground for antisocial behavior. Similarly, some other online outlets also enable the spread of antisocial semantics that plague the idea of freedom of speech online. It obstructs constructive discussion and leads to many users abandoning participation. At this stage, most of these platforms rely on users reporting such antisocial behavior to these platforms instead of automated detection, which is imperative if the prevention has to work on scale. These platforms may have some measures in place to prevent antisocial behavior online. However, these should be more effective. We, in this research, proposed an approach based on natural language processing and deep learning techniques that can enable online platforms to detect and restrict antisocial behavior proactively. As can be seen, by our results, our model can detect antisocial behavior on Twitter with very high accuracy. The model can be integrated into an online system to depict such behavior on a live data stream. Once detected, appropriate action can be taken, such as deleting the tweet or blocking the user to prevent future incidents.
In this study, we have explored data mainly from Twitter. Further studies can be conducted by collecting data from various online platforms. The diversity of data used will enable models to learn and perform better. Furthermore, we would like to explore other personality and behavior disorders that fall under the same category as antisocial behavior. Diagnostic criteria for these disorders overlap in some instances and can present a challenge in training a model to classify and distinguish these disorders with high accuracy and precision. Future work can also be done by categorizing tweets into different antisocial behaviors and contexts. Depending on the seriousness of the situation, this may lead to offering help to victims by notifying authorities of the dire circumstances. Despite the findings and results, our work has a few limitations. The size of our data set is moderate due to the labor-intensive job of manually labeling tweets. It consists of around 55,000 tweets; however, a larger dataset could have bought in more diversity regarding the feature words and phrases that our algorithms used to learn from. Secondly, we used around 30 different terms to search for tweets on Twitter.
Once these tweets were collected, they were labeled antisocial or non-antisocial. In our subsequent study, we would like to increase the number of these phrases to at least 100. This will bring more diversity to words, phrases, contexts, semantics, and scenarios used to train our classifier. Nevertheless, the findings and the results are valuable in guiding further antisocial behavior studies from social media data using a deep learning approach.