Sentiment Analysis of Twitter about ChatGPT

Document Type : Original Article

Authors
Ameneh Khadivar 1
Paria Oman 2
Fatemeh Abbasi 3
1 Associate Prof., Department of Management, Faculty of Social Sciences and Economics, Alzahra University, Tehran, Iran
2 MSc., Department of Management, Faculty of Social Sciences and Economics, Alzahra University, Tehran, Iran
3 Assistant Prof., Industrial Management and IT, Faculty of Management and Accounting, Shahid Beheshti University, Tehran, Iran
10.22034/aimj.2024.431651.1576
Abstract
In recent years, we have witnessed significant advancements in artificial intelligence across many aspects of human life. One way AI can enhance human life is through the use of chatbots. A chatbot that has recently been introduced with much attention and is promised to revolutionize the way people interact with technology is ChatGPT. However, with the widespread use of AI chatbots, concerns about data privacy and security have emerged. Evaluating these concerns can offer insights into public perceptions and help improve data privacy policies. Previous research on this technology has mainly focused on its technical aspects, whereas understanding public sentiment about ChatGPT as a transformative technology can provide insights into its potential success or failure, as well as its strengths and weaknesses. In line with this, the present study aims to examine the perceptions of Twitter users regarding ChatGPT through sentiment analysis and topic modeling. A total of 478,266 tweets were collected via the official Twitter API, and following sentiment analysis using the BERT model—one of the advanced algorithms in deep learning—the results showed an accuracy of 82%. Additionally, through topic modeling using the BERTopic algorithm, based on BERT, the results achieved a coherence (C_V) score of 0.632 and a U_Mass score of -2.957. According to the study’s findings, the nine most discussed topics among Twitter users are: artificial intelligence, search engines, future jobs, answering questions, education, programming, large language models, business, and healthcare. The results indicate that users expressed the highest percentage of positive sentiment towards the topics of large language models, education, and business, while the most negative sentiments were expressed regarding future jobs, healthcare, and artificial intelligence. After neutral opinions, which made up the largest portion of the data, positive tweets significantly outnumbered negative ones, reflecting the public’s satisfaction and optimism towards ChatGPT technology.

Keywords

Ante, L. & Demir, E. (2023). The ChatGPT Effect on AI-themed cryptocurrencies. Available at SSRN 4350557.
 
Bello, A., Ng, S.C. & Leung, M.-F. (2023). A BERT framework to sentiment analysis of tweets. Sensors, 23(1), 506.
Bian, J., Yoshigoe, K., Hicks, A., Yuan, J., He, Z., Xie, M., Guo, Y., Prosperi, M., Salloum, R. & Modave, F. (2016). Mining Twitter to assess the public perception of the “Internet of Things”. PloS one, 11(7), e0158450.
Bostrom, N. & Yudkowsky, E. (2018). The ethics of artificial intelligence. In Artificial intelligence safety and security (pp. 57-69). Chapman and Hall/CRC.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G. & Askell, A. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
Chen, W., Rabhi, F., Liao, W. & Al-Qudah, I. (2023). Leveraging State-of-the-Art Topic Modeling for News Impact Analysis on Financial Markets: A Comparative Study. Electronics, 12(12), 2605.
Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V. & Ahuja, M. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642.
George, A. S. & George, A. H. (2023). A review of ChatGPT AI's impact on several business sectors. Partners Universal International Innovation Journal, 1(1), 9-23.
Greenhill, A. T. & Edmunds, B. R. (2020). A primer of artificial intelligence in medicine. Techniques and Innovations in Gastrointestinal Endoscopy, 22(2), 85-89.
Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794.
Haleem, A., Javaid, M. & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil transactions on benchmarks, standards and evaluations, 2(4), 100089.
Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N. & Ahmad, H. (2022). " I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data. arXiv preprint arXiv:2212.05856.
Haque, R., Ramadurai, A., Hasanuzzaman, M. & Way, A. (2019). Mining purchase intent in twitter. Computación y Sistemas, 23(3), 871-881.
Islam, I. & Islam, M. N. (2023). Opportunities and Challenges of ChatGPT in Academia: A Conceptual Analysis. Authorea Preprints.
Jalil, S., Rafi, S., LaToza, T. D., Moran, K. & Lam, W. (2023). Chatgpt and software testing education: Promises & perils. arXiv preprint arXiv:2302.03287.
Kaul, V., Enslin, S. & Gross, S. A. (2020). History of artificial intelligence in medicine. Gastrointestinal endoscopy, 92(4), 807-812.
King, M. R. & ChatGPT. (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher education. Cellular and Molecular Bioengineering, 16(1), 1-2.
Leiter, C., Zhang, R., Chen, Y., Belouadi, J., Larionov, D., Fresen, V. & Eger, S. (2023). Chatgpt: A meta-analysis after 2.5 months. arXiv preprint arXiv:2302.13795.
Lucy, L. & Bamman, D. (2021). Gender and representation bias in GPT-3 generated stories. Proceedings of the Third Workshop on Narrative Understanding. (pp. 48-55).
Macdonald, C., Adeloye, D., Sheikh, A. & Rudan, I. (2023). Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. Journal of global health, 13.
Malik, P., Pathania, M. & Rathaur, V.K. (2019). Overview of artificial intelligence in medicine. Journal of family medicine and primary care, 8(7), 2328.
Mitrović, S., Andreoletti, D. & Ayoub, O. (2023). Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text. arXiv preprint arXiv:2301.13852.
Nagarhalli, T. P., Vaze, V. & Rana, N. (2020). A review of current trends in the development of chatbot systems. 2020 6th International conference on advanced computing and communication systems (ICACCS).
Novendri, R., Callista, A. S., Pratama, D. N. & Puspita, C. E. (2020). Sentiment analysis of YouTube movie trailer comments using naïve bayes. Bulletin of Computer Science and Electrical Engineering, 1(1), 26-32.
Pavlik, J. V. (2023). Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. Journalism & Mass Communication Educator, 10776958221149577.
Rahaman, M. (2023). Can chatgpt be your friend? emergence of entrepreneurial research. Emergence of Entrepreneurial Research (February 18, 2023).
Rathore, B. (2022). Supply Chain 4.0: Sustainable Operations in Fashion Industry. International Journal of New Media Studies (IJNMS), 9(2), 8-13.
Rathore, B. (2023). Future of AI & Generation Alpha: ChatGPT beyond Boundaries. Eduzone: International Peer Reviewed/Refereed Multidisciplinary Journal, 12(1), 63-68.
Rudolph, J., Tan, S. & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1).
Shahriar, S. & Hayawi, K. (2023). Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations. arXiv preprint arXiv:2302.13817.
Taecharungroj, V. (2023). “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter. Big Data and Cognitive Computing, 7(1), 35.
Thorp, H. H. (2023). ChatGPT is fun, but not an author. In (Vol. 379, pp. 313-313): American Association for the Advancement of Science.
Zarifhonarvar, A. (2023). Economics of chatgpt: A labor market view on the occupational impact of artificial intelligence. Available at SSRN 4350925.
Zhang, C. & Lu, Y. (2021). Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration, 23, 100224.
Zhang, C., Xu, X. & Chen, H. (2020). Theoretical foundations and applications of cyber-physical systems: a literature review. Library Hi Tech, 38(1), 95-104.
Zhong, Q., Ding, L., Liu, J., Du, B. & Tao, D. (2023). Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198.
Iranian Journal of Information Management
Volume 9, Issue 1 - Serial Number 16
September 2024
Pages 159-182

  • Receive Date 22 December 2023
  • Revise Date 20 September 2024
  • Accept Date 03 November 2024