1. bookVolume 2020 (2020): Issue 4 (October 2020)
Journal Details
License
Format
Journal
First Published
16 Apr 2015
Publication timeframe
4 times per year
Languages
English
access type Open Access

Effective writing style transfer via combinatorial paraphrasing

Published Online: 17 Aug 2020
Page range: 175 - 195
Received: 29 Feb 2020
Accepted: 16 Jun 2020
Journal Details
License
Format
Journal
First Published
16 Apr 2015
Publication timeframe
4 times per year
Languages
English

Stylometry can be used to profile or deanonymize authors against their will based on writing style. Style transfer provides a defence. Current techniques typically use either encoder-decoder architectures or rule-based algorithms. Crucially, style transfer must reliably retain original semantic content to be actually deployable. We conduct a multifaceted evaluation of three state-of-the-art encoder-decoder style transfer techniques, and show that all fail at semantic retainment. In particular, they do not produce appropriate paraphrases, but only retain original content in the trivial case of exactly reproducing the text. To mitigate this problem we propose ParChoice: a technique based on the combinatorial application of multiple paraphrasing algorithms. ParChoice strongly outperforms the encoder-decoder baselines in semantic retainment. Additionally, compared to baselines that achieve nonnegligible semantic retainment, ParChoice has superior style transfer performance. We also apply ParChoice to multi-author style imitation (not considered by prior work), where we achieve up to 75% imitation success among five authors. Furthermore, when compared to two state-of-the-art rule-based style transfer techniques, ParChoice has markedly better semantic retainment. Combining ParChoice with the best performing rulebased baseline (Mutant-X [34]) also reaches the highest style transfer success on the Brennan-Greenstadt and Extended-Brennan-Greenstadt corpora, with much less impact on original meaning than when using the rulebased baseline techniques alone. Finally, we highlight a critical problem that afflicts all current style transfer techniques: the adversary can use the same technique for thwarting style transfer via adversarial training. We show that adding randomness to style transfer helps to mitigate the effectiveness of adversarial training.

Keywords

[1] Ahmed Abbasi and Hsinchun Chen. Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace. ACM Transactions on Information and System Security, 26(2):1–29, 2008.Search in Google Scholar

[2] Sadia Afroz, Michael Brennan, and Rachel Greenstadt. Detecting Hoaxes, Frauds, and Deception in Writing Style Online. In Proceedings of the 2012 IEEE Symposium on Security and Privacy, pages 461–475, 2012.Search in Google Scholar

[3] Mishari Almishari, Ekin Oguz, and Gene Tsudik. Fighting Authorship Linkability with Crowdsourcing. In Proceedings of the second ACM conference on Online social networks, pages 69–82, 2014.Search in Google Scholar

[4] Douglas Bagnall. Author identification using multi-headed recurrent neural networks. In Working Notes of CLEF 2015 - Conference and Labs of the Evaluation Forum, 2015.Search in Google Scholar

[5] Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, 2005.Search in Google Scholar

[6] Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly, Beijing, 2009.Search in Google Scholar

[7] Michael Brennan, Sadia Afroz, and Rachel Greenstadt. Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity. ACM Transactions on Information and System Security, 15(3):12:1–12:22, 2011.Search in Google Scholar

[8] Michael Brennan and Rachel Greenstadt. Practical Attacks Against Authorship Recognition Techniques. In Proceedings of the Twenty-First Conference on Innovative Applications of Artificial Intelligence, pages 60–65, 2009.Search in Google Scholar

[9] Marcelo Luiz Brocardo, Issa Traore, Isaac Woungang, and Mohammad S. Obaidat. Authorship verification using deep belief network systems. Communication Systems, 30(12), 2017.Search in Google Scholar

[10] Aylin Caliskan and Rachel Greenstadt. Translate once, translate twice, translate thrice and attribute: Identifying authors and machine translation tools in translated text. In Proceedings of the 2012 IEEE Sixth International Conference on Semantic Computing (ICSC), pages 121–125, 2012.Search in Google Scholar

[11] Andrew Carnie. Constituent Structure. Oxford University Press, Oxford, 2008.Search in Google Scholar

[12] Daniel Castro-Castro, Reynier Ortega Bueno, and Rafael Mu noz. Author masking by sentence transformation – notebook for PAN at CLEF. In Working notes of CLEF2017, 2017.Search in Google Scholar

[13] C.K. Chow. On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory, 16:41–46, 1970.Search in Google Scholar

[14] Siobahn Day, James Brown, Zachery Thomas, India Gregory, Lowell Bass, and Gerry Dozier. Adversarial Authorship, AuthorWebs, and entropy-based evolutionary clustering. In Proceedings of the 25th International Conference on Computer Communication and Networks (ICCCN), pages 1–6, 2016.Search in Google Scholar

[15] Iqbal Farkhund, Hamad Binsalleeh, Benjamin C.M. Fung, and Mourad Debbabi. Mining writeprints from anonymous e-mails for forensic investigation. Digital Investigation, 7(1–2):56–64, 2013.Search in Google Scholar

[16] César Ferri, Peter Flach, and José Hernández-Orallo. Delegating classifiers. In Proceedings of the 21th International Conference on Machine Leaning (ICML’04), pages 106–110, 2004.Search in Google Scholar

[17] Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. Style transfer in text: Exploration and evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 663–670, 2018.Search in Google Scholar

[18] Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. PPDB: The paraphrase database. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 758–764, 2013.Search in Google Scholar

[19] Zhenhao Ge and Yufang Sun. Domain specific author attribution based on feedforward neural network language models. In Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods (ICPRAM), pages 597–604, 2016.Search in Google Scholar

[20] David Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston, 1989.Search in Google Scholar

[21] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS), pages 2672–2680, 2014.Search in Google Scholar

[22] Jack Grieve. Quantitative authorship attribution: An evaluation of techniques. Literary and Linguistic Computing, 22(3):251–270, 2007.Search in Google Scholar

[23] Tommi Gröndahl and N. Asokan. EAT2seq: A generic framework for controlled sentence transformation without taskspecific training. arXiv preprint arXiv: 1902.09381, 2019.Search in Google Scholar

[24] Tommi Gröndahl and N. Asokan. Text analysis in adversarial settings: Does deception leave a stylistic trace? ACM Computing Surveys, 52(3):45:1–45:36, 2019.Search in Google Scholar

[25] Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N. Asokan. All you need is “love”: Evading hate speech detection. In Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security (AISec’11), pages 2–12, 2018.Search in Google Scholar

[26] Michael Haardt. GNU Diction, 2005 (accessed February 24, 2020). https://www.gnu.org/software/diction/.Search in Google Scholar

[27] Thanh Nghia Ho and Wee Keong Ng. Application of stylometry to DarkWeb forum user identification. In Proceedings of Information and Communications Security, pages 173–183, 2016.Search in Google Scholar

[28] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997.Search in Google Scholar

[29] Charles D. Hollingsworth. Syntactic Stylometry: Using Sentence Structure for Authorship Attribution. PhD thesis, University of Georgia, 2012.Search in Google Scholar

[30] Charles D. Hollingsworth. Using dependency-cased annotations for authorship identification. In Proceedings of the International Conference on Text, Speech and Dialogue, pages 314–319, 2012.Search in Google Scholar

[31] Matthew Honnibal and Mark Johnson. An improved nonmonotonic transition system for dependency parsing. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 1373–1378, 2015.Search in Google Scholar

[32] Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, 2017.Search in Google Scholar

[33] Patrick Juola. Large-scale experiments in authorship attribution. English Studies, 93(3):275–283, 2012.Search in Google Scholar

[34] Patrick Juola. Stylometry and immigration: A case study. Journal of Law and Policy, 21(2):287–298, 2013.Search in Google Scholar

[35] Mika Juuti, Bo Sun, Tatsuya Mori, and N. Asokan. Stay ontopic: Generating context-specific fake restaurant reviews. In Proceedings of the 23rd European Symposium on Research in Computer Security (ESORICS), pages 132–151, 2018.Search in Google Scholar

[36] Georgi Karadzhov, Tsvetomila Mihaylova, Yasen Kiprov, Georgi Georgiev, Ivan Koychev, and Preslav Nakov. The case for being average: A mediocrity approach to style masking and author obfuscation. In Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF), pages 173–185, 2017.Search in Google Scholar

[37] Foaad Khosmood. Comparison of sentence-level paraphrasing approaches for statistical style transformation. In Proceedings of the International Conference on Artificial Intelligence, 2012.Search in Google Scholar

[38] Foaad Khosmood and Robert Levinson. Automatic natural language style classification and transformation. In Proceedings of the 2008 BCS-IRSG Conference on Corpus Profiling, page 3, 2008.Search in Google Scholar

[39] Foaad Khosmood and Robert Levinson. Toward automated stylistic transformation of natural language text. In Proceedings of the Digital Humanities, pages 177–181, 2009.Search in Google Scholar

[40] Foaad Khosmood and Robert Levinson. Automatic synonym and phrase replacement show promise for style transformation. In Proceedings of The Ninth International Conference on Machine Learning and Applications, pages 958–961, 2010.Search in Google Scholar

[41] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pages 2278–2324, 1998.Search in Google Scholar

[42] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text againstreal-world applications. In Proceedings of Network and Distributed Systems Security (NDSS), 2019.Search in Google Scholar

[43] Lajanugen Logeswaran, Honglak Lee, and Samy Bengio. Content preserving text generation with attribute controls. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS), 2018.Search in Google Scholar

[44] Nathan Mack, Jasmine Bowers, Henry Williams, Gerry Dozier, and Joseph Shelton. The best way to a strong defense is a strong offense: Mitigating deanonymization attacks via iterative language translation. International Journal of Machine Learning and Computing, 5(5):409–413, 2015.Search in Google Scholar

[45] Asad Mahmood, Faizan Ahmad, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. A girl has no name: Automated authorship obfuscation using Mutant-X. In Proceedings on Privacy Enhancing Technologies (PETS), pages 54–71, 2019.Search in Google Scholar

[46] Muharram Mansoorizadeh, Taher Rahgooy, Mohammad Aminiyan, and Mahdy Eskandari. Author obfuscation using WordNet and language models – notebook for PAN at CLEF 2016. In CLEF 2016 Evaluation Labs and Workshop – Working Notes Papers, 2016.Search in Google Scholar

[47] Andrew W.E. McDonald, Sadia Afroz, Aylin Caliskan, Ariel Stolerman, and Rachel Greenstadt. Use fewer instances of the letter i: Toward writing style anonymization. In Simone Fischer-Hübner and Matthew Wright, editors, Privacy Enhancing Technologies. Volume 7384 of Lecture Notes in Computer Science, pages 299–318. 2012.Search in Google Scholar

[48] Tsvetomila Mihaylova, Georgi Karadjov, Preslav Nakov, Yasen Kiprov, Georgi Georgiev, and Ivan Koychev. SU@PAN’2016: Author obfuscation – notebook for PAN at CLEF 2016. In CLEF 2016 Evaluation Labs and Workshop – Working Notes Papers, 2016.Search in Google Scholar

[49] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS), page 3111–3119, 2013.Search in Google Scholar

[50] George A. Miller. WordNet: A lexical database for English. Communications of the ACM, 38(11):39–41, 1995.Search in Google Scholar

[51] Arvind Narayanan, Hristo Paskov, Neil Zhenqiang Gong, John Bethencourt, Emil Stefanov, Eui Chul Richard Shin, and Dawn Song. On the feasibility of internet-scale author identification. In Proceedings of the IEEE Symposium on Security and Privacy, pages 300–314, 2012.Search in Google Scholar

[52] Roberto Navigli. Word sense disambiguation: A survey. ACM Computing Surveys, 41(2):1–69, 2009.Search in Google Scholar

[53] Rebekah Overdorf and Rachel Greenstadt. Blogs, twitter feeds, and reddit comments: Cross-domain authorship attribution. In Proceedings on Privacy Enhancing Technologies (PETS), pages 155–171, 2016.Search in Google Scholar

[54] Lluís Padró and Evgeny Stanilovsky. Freeling 3.0: Towards wider multilinguality. In Proceedings of the Language Resources and Evaluation Conference (LREC 2012), pages 2473—-479, 2012.Search in Google Scholar

[55] Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers), pages 425–430, 2015.Search in Google Scholar

[56] Martin Potthast, Sarah Braun, Tolga Buz, Fabian Duffhauss, Florian Friedrich, Jörg Marvin Gülzow, Jakob Köhler, Winfried Lötzsch, Fabian Müller, Maike Elisa Müller, Robert Paßmann, Bernhard Reinke, Lucas Rettenmeier, Thomas Rometsch, Timo Sommer, Michael Träger, Sebastian Wilhelm, Benno Stein, Efstathios Stamatatos, and Matthias Hagen. Who wrote the web? Revisiting influential author identification research applicable to information retrieval. In Nicola Ferro, Fabio Crestani, Marie-Francine Moens, Josiane Mothe, Fabrizio Silvestri, Giorgio Maria Di Nunzio, Claudia Hauff, and Gianmaria Silvello, editors, Advances in Information Retrieval, pages 393–407. Springer International Publishing, 2016.Search in Google Scholar

[57] Martin Potthast, Matthias Hagen, and Benno Stein. Author obfuscation: Attacking the state of the art in authorship verification. In CLEF 2016 Working Notes, 2016.Search in Google Scholar

[58] Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W. Black. Style Transfer Through Back-Translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 866–876, 2018.Search in Google Scholar

[59] Daniel Preotiuc-Pietro, Jordan Carpenter, and Lyle Ungar. Personality driven differences in paraphrase preference. In Proceedings of the Second Workshop on Natural Language Processing and Computational Social Science, pages 17–26, 2017.Search in Google Scholar

[60] Press Freedom Index. RSF Index 2018: Hatred of journalism threatens democracies. https://rsf.org/en/rsf-index-2018-hatred-journalism-threatens-democracies (May 1st 2018).Search in Google Scholar

[61] Sravana Reddy and Kevin Knight. Obfuscating gender in social media writing. In Proceedings of Workshop on Natural Language Processing and Computational Social Science, pages 17–26, 2016.Search in Google Scholar

[62] Jonathan Schler, Moshe Koppel, Shlomo Argamon, and James Pennebaker. Effects of age and gender on blogging. In Proceedings of AAAI Spring Symposium on Computational Approaches for Analyzing Weblogs, 2006.Search in Google Scholar

[63] Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from non-parallel text by crossalignment. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS), 2017.Search in Google Scholar

[64] Rakshith Shetty, Bernt Schiele, and Mario Fritz. A4NT: Author attribute anonymity by adversarial training of neural machine translation. In Proceedings of the 27th USENIX Security Symposium, pages 1633–1650, 2018.Search in Google Scholar

[65] Efstathios Stamatatos. A survey of modern authorship attribution methods. Journal of the American Society for Information Science and Technology, 60(3):538–556, 2009.Search in Google Scholar

[66] Ariel Stolerman, Rebekah Overdorf, Sadia Afroz, and Rachel Greenstadt. Breaking the closed-world assumption in stylometric authorship attribution. In Gilbert Peterson and Sujeet Shenoi, editors, Advances in Digital Forensics X, pages 185–205, Berlin, Heidelberg, 2014. Springer Berlin Heidelberg.Search in Google Scholar

[67] Kl Surendran, O.P. Harilal, Hrudya Poroli, Prabaharan Poornachandran, and N.K. Suchetha. Stylometry detection using deep learning. In Proceedings of Computational Intelligence in Data Mining, pages 749–757, 2017.Search in Google Scholar

[68] Neal Tempestt, Kalaivani Sundararajan, Aneez Fatima, Yiming Yan, Yingfei Xiang, and Damon Woodard. Surveying stylometry techniques and applications. ACM Computing Surveys, 50(6), 2017.Search in Google Scholar

[69] Louis Tesnière. Èléments de syntaxe structurale. Klincksieck, Paris, 1959.Search in Google Scholar

[70] Jingjing Xu, Xu Sun, Qi Zeng, Xuancheng Ren, Xiaodong Zhang, Houfeng Wang, and Wenjie Li. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of the Association for Computational Linguistics (ACL), pages 979–988, 2018.Search in Google Scholar

[71] Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. Unsupervised text style transfer using language models as discriminators. In Proceedings of the International Conference on Neural Information Processing (NIPS), pages 7287–7298, 2018.Search in Google Scholar

[72] Liang-Chih Yu, Jin Wang, K. Robert Lai, and Xuejie Zhang. Refining word embeddings using intensity scores for sentiment analysis. In IEEE/ACM Transactions on Audio, Speech, and Language Processing, pages 671–681, 2018.Search in Google Scholar

[73] Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, and Yann LeCun. Adversarially regularized autoencoders. In Proceedings of the 35th International Conference on Machine Learning, pages 5902–5911, 2018.Search in Google Scholar

[74] Rong Zheng, Jiexun Li, Hsinchun Chen, and Zan Huang. A framework of authorship identification for online messages: Writing style features and classification techniques. Journal of the American Society for Information Science and Technology, 57(3):378–393, 2006.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo