Sentence Fusion using Deep Learning
DOI:
https://doi.org/10.4108/eetiot.4605Keywords:
Abstractive Summarization, Deep Learning, Sentence FusionAbstract
The human process of document summarization involves summarizing a document by sentence fusion. Sentence fusion combines two or more sentences to create an abstract sentence. Sentence fusion is useful to convert an extractive summary to an abstractive summary. The extractive summary contains a set of salient sentences selected from a single document or multiple related documents. Redundancy creates problems while creating an extractive summary because it contains sentences whose segments or phrases are redundant. Sentence fusion helps to remove redundancy by fusing sentences into a single abstract sentence. This moves an extractive summary to an abstractive summary. In this paper, we present an approach that uses a deep learning model for sentence fusion. which is trained over a large dataset. We have tested our approach through both manual evaluation and system evaluation. The result of our proposed approach shows that our model is good enough to fuse sentences effectively.
Downloads
References
Liao, K., Lebanoff, L., Liu, F.: Abstract meaning representation for multi-document summarization, In: Proceedings of the 27th International Conference on Computational Linguistics, COLING, Santa Fe, New Mexico, USA, pp. 1178–1190.(2018).
Chenal, V., Cheung, J.C.K.: Predicting sentential semantic compatibility for aggregation in text-to-text generation, In 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Osaka, Japan, pp. 1016-1070. (2016).
Barzilay, R., McKeown, K.R.: Sentence fusion for multidocument news summarization, Computer Linguist. vol. 31 (3), pp. 297–328, (2005). DOI: https://doi.org/10.1162/089120105774321091
Durrett, G., Berg-Kirkpatrick T., Klein, D.: Learning-based single-document summarization with compression and anaphoricity constraints”, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, Vol. 1, pp.1998–2008. August,(2016). DOI: https://doi.org/10.18653/v1/P16-1188
Bing, L. et al.: Abstractive multi-document summarization via phrase selection and merging, In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Beijing, China, Vol. 1, pp.1587–1597.(2015). DOI: https://doi.org/10.3115/v1/P15-1153
Martins, A.F.T., Smith, N.A.: Summarization with a joint model for sentence extraction and compression, In: Proceedings of the Workshop on Integer Linear Programming for Natural Langauge Processing, In ILP, Association for Computational Linguistics, Stroudsburg, PA, USA, pp. 1–9.(2009). DOI: https://doi.org/10.3115/1611638.1611639
Chen Y., Bansal, M.: Fast abstractive summarization with reinforce-selected sentence rewriting, In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, vol. 1, pp. 675–686.(2018).
Mendes, M. et al.: Jointly extracting and compressing documents with summary state representations, In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 3955–3966.(2019). DOI: https://doi.org/10.18653/v1/N19-1397
Thadani, K., McKeown, K.: Sentence Compression with Joint Structural Inference, In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning, Sofia, Bulgaria, pp. 65-74.(2013).
Marsi, E., Krahmer, E.: Classification of Semantic Relations by Humans and Machines, In: Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment, Association for Computational Linguistics, pp. 1-6.(2005). DOI: https://doi.org/10.3115/1631862.1631863
Filippova, K., Strube, M.: Dependency Tree Based Sentence Compression, In: Proceedings of the Fifth International Natural Language Generation Conference, Association for Computational Linguistics, pp. 25-32.(2008). DOI: https://doi.org/10.3115/1708322.1708329
Cheung, J., Penn, G.: Unsupervised Sentence Enhancement for Automatic Summarization, In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, Association for Computational Linguistics, pp. 775-786.(2014). DOI: https://doi.org/10.3115/v1/D14-1085
Gerani, S. et al: Abstractive Summarization of Product Reviews Using Discourse Structure, In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP),Doha, Qatar, Association for Computational Linguistics, pp. 1602-1613,(2014). DOI: https://doi.org/10.3115/v1/D14-1168
Mehdad, Y., Carenini, G., Tompa, F.W., NG, T.R.: Abstractive Meeting Summarization with Entailment and Fusion, In: Proceedings of the 14th European Work-shop on Natural Language Generation, Sofia, Bulgaria, Association for Computational Linguistics, pp. 136-146.(2013).
Liu, F. et al: Toward Abstractive Summarization Using Semantic Representations, In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ”Denver, Colorado,Association for Computational Linguistics, pp. 1077-1086.(2015). DOI: https://doi.org/10.3115/v1/N15-1114
Nayeem, M., Fuad, T., Chali, Y.: Abstractive Unsupervised Multi-Document Summarization using Paraphrastic Sentence Fusion, In: Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA, Association for Computational Linguistics”,pp. 1191-1204.(2018).
Lebanoff, L. et al.: Analyzing Sentence Fusion in Abstractive Summarization, In: Proceedings of the 2nd Workshop on New Frontiers in Summarization,Hong Kong, China,Association for Computational Linguistics”, pp. 104-110.(2019). DOI: https://doi.org/10.18653/v1/D19-5413
Erkan, G., Radev, D. R.: LexRank: Graph-Based Lexical Centrality as Salience in Text Summarization, AI Access Foundation, Vol. 22(1), pp. 457-479.(2004). DOI: https://doi.org/10.1613/jair.1523
Cao, Z., Wei, F., Li, W., Li, S.: Faithful to the original: Fact aware neural abstractive summarization, In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), (2018). DOI: https://doi.org/10.1609/aaai.v32i1.11912
Song, K., Zhao, L., Liu, F.: Structure-infused copy mechanisms for abstractive summarization, In: Proceedings of the International Conference on Computational Linguistics (COLING),(2018).
See, A., Liu, P.J, Manning, C.D.: Get to the point: Summarization with pointer-generator networks, In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), (2017). DOI: https://doi.org/10.18653/v1/P17-1099
Celikyilmaz, A., Bosselut, A., Xiaodong, H., Yejin, C.: Deep Communicating Agents for Abstractive Summarization, In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, pp. 1662-1675.(2018). DOI: https://doi.org/10.18653/v1/N18-1150
Raffel, C. et al.: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,vol. 21,(2022).
Lewis, M.: BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension, In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, pp. 7871-7880.(2020). DOI: https://doi.org/10.18653/v1/2020.acl-main.703
Jiwei, T., Xiaojun, W., Jianguo, X.: Abstractive document summarization with a graph-based attentional neural model, In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL),(2017).
Gehrmann, S., Deng, Y., Rush, A.: Bottom-Up Abstractive Summarization, In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 4098-4109.(2018). DOI: https://doi.org/10.18653/v1/D18-1443
Chen, Y., Bansal, M.: Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting, pp. 675-686.(2018). DOI: https://doi.org/10.18653/v1/P18-1063
Lebanoff, L. , Song, K., Liu, F.: Adapting the Neural Encoder-Decoder Frame-work from Single to Multi-Document Summarization, In :Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 4131-4141.(2018). DOI: https://doi.org/10.18653/v1/D18-1446
Yang, L., Lapata, L.: Text Summarization with Pretrained Encoders, In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019, Hong Kong, China, Association for Computational Linguistics, pp. 3728-3738.(2019).
Callison, C., Osborne, M., Koehn, P.: Re-evaluating the role of BLEU in machine translation research, In: 11th conference of the european chapter of the association for computational linguistics, pp. 249-256. (2006).
Papineni, K., Roukos, S., Ward, T., Zhu, W.Z.: BLEU: A Method for Automatic Evaluation of Machine Translation, In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 311-318.(2002). DOI: https://doi.org/10.3115/1073083.1073135
Lin, C.: ROUGE: A Package for Automatic Evaluation of Summaries, In: Text Summarization Branches Out, Association for Computational Linguistics, pp. 74-81.(2004).
Wan X., Yang, J.: Improved Affinity Graph Based Multi-Document Summarization, In: Proceedings of the Human Language Technology Conference of the NAACL, Association for Computational Linguistics, pp. 181-184. (2006). DOI: https://doi.org/10.3115/1614049.1614095
Zolotareva, E., Tashu, T.M., Horv ́ath, T.: Abstractive Text Summarization using Transfer Learning,(2020).
Fatih, E., Guven, F., Galip, A.: Turkish abstractive text document summarization using text to text transfer transformer, Alexandria Engineering Journal, vol. 68, pp. 1-13.(2023). DOI: https://doi.org/10.1016/j.aej.2023.01.008
Sarkar, K.: Syntactic trimming of extracted sentences for improving extractive multi-document summarization, Journal of Computing, vol. 2, pp. 177-184.(2010).
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
