It has always bothered me that there seems to be a serious under-representation of women who are involved in the development of machine translation (MT). Since it didn’t make much sense for me, a man, to write and complain about that, I asked three women—Lynne Bowker, Sharon O’Brien, and Vassilina Nikoulina—who are involved in MT in academics and development to discuss the topic with me. It ended up being a phenomenal exchange (with me on the receiving rather than the giving side). Without further ado, here it is.
Jost: I’m so glad that the three of you are willing to talk with me about women in MT development and women in the academic pursuit of MT. But first, would you mind introducing yourselves?
Lynne: I grew up in Canada and received my initial BA in translation. I’m a certified French>English translator (Association of Translators and Interpreters of Ontario) and worked briefly for the Canadian government within the Translation Bureau and as a freelancer before studying for my graduate degree. I received an MA in translation from the University of Ottawa in the early 1990s, when I became interested in technologies for terminologists and translators, which were just beginning to emerge on the market. I continued my studies in language engineering and earned a PhD from the University of Manchester Institute of Science and Technology. My first academic job was at Dublin City University in Ireland, where I taught both translation and computational linguistics. In 2000, I returned to Canada and am currently a full professor at the University of Ottawa with a cross-appointment between the School of Translation and Interpretation and the School of Information Studies.
Jost: Let me add one more thing to that, Lynne. You were also recently chosen as a Fellow of the Royal Society of Canada. Congratulations!
Sharon: I received a BA in applied languages (with French and German) from Dublin City University, in what was effectively a translator training program. In my final year, the instructors introduced us to the translation system from ALPNET, after which I was hooked on translation technology! I had the opportunity to do a master’s research project with the Eurotra project, an ambitious machine translation project established and funded by the European Commission from 1978 to 1992, where I investigated the effectiveness of MT for “sublanguage” (or language for special purposes). Afterwards, I moved to Luxembourg for a three-month internship with the European Parliament. During this time, I had the pleasure of using the DOS version of the Trados Alignment tool for three whole months (i.e., T-Align, the precursor to WinAlign) to align translations in 11 languages of the CVs of members of Parliament so the translation memories (TM) could be used for the upcoming elections.
Somehow, this qualified me as a “language technology specialist” for a localization company in Dublin that was interested in introducing Trados into its workflow. My job was to define the localization processes with a TM tool and train and support the translators. At one point, a very adventurous client wanted to test MT (which was still known as rule-based MT at the time), so I was responsible for that project. After a few years of working in the localization sector, I decided the time was right to get a PhD and that MT and post-editing was an interesting topic. I went back to Dublin City University and completed that PhD in 2006, after which I became a faculty member within the university’s School of Applied Language and Intercultural Studies.
Vassilina: I grew up in Russia where I completed my initial studies in applied mathematics. I arrived in France at the École Polytechnique in 2003, where I completed the equivalent of a master’s degree in computer science. I got acquainted with MT during my first internship at Systran in 2005. Despite my mathematical background, I’ve always been interested in foreign language studies, which is why I decided to pursue research in MT. I completed a PhD focused around statistical MT at the University of Grenoble and Xerox Research Center in Europe (XRCE) in 2010. Afterward, I kept working on MT and other topics related to natural language processing within Xerox Research in Grenoble. In 2017, Naver, Korea’s leading internet portal company, acquired XRCE. As a follow-up of this acquisition, I had a chance to spend 10 months with the team working on Papago1 (Naver’s MT engine) in Korea in 2019. It was a great and challenging experience that allowed me to transition from the MT research world to its practical application, and at the same time act as an active MT user in my everyday life. (I used Papago MT every day, several times a day!) I’m now back in France conducting my usual research activities, but I try to keep in touch with Papago colleagues and collaborate on different MT-related subjects.
J: Great. Let’s start then. Do you think it’s true that there is a difference between the number of women working with MT in academia and industry?
L: This seems a bit anecdotal. Are there any statistics on how many women work with MT in academia/industry versus how many men? Unfortunately, I don’t know of any statistics, and I really don’t have a good grip on the nature of the gender imbalance other than in the most vague or general way. Also, “working with MT” covers a lot of ground—user versus developer and everything in between.
J: Thanks for pointing that out. I checked with the group program manager for MT at Microsoft and he pointed me to a document that showed that of all the technical roles at Microsoft, approximately 20% are held by women.2 He confirmed that this percentage is approximately the same for MT development. And it’s indeed “MT development” that I had hoped to focus on in our discussion, but if you have insights on different kinds of usage between female and male users, I would be very interested in that as well.
L: As I mentioned earlier, I work at the University of Ottawa’s School of Translation and Interpretation, where one of our goals is to educate language professionals in the use of translation tools (both computer-assisted translation and MT). So, most of my MT-related attention focuses on the user perspective. I don’t work directly in tool development per se, and the students I work with don’t usually go on to work in development either. Like me, they are language professionals and language researchers who have an interest in how translation technology is used by language professionals and others. In my experience, both in Dublin and in Ottawa, the development work is more often being done by researchers in computer science departments. Of course, there’s conversation between the translation school and the computer science department, but the development work is principally driven by the computer science researchers. Here in Ottawa, we also have the National Research Council of Canada, which has a very active research and development team working on language technologies, including MT (e.g., the Portage system).
Although there are some women researchers in both the university computer science department and at the National Research Council of Canada, they are certainly in the minority. In contrast, women are in the majority at the School of Translation and Interpretation, particularly in the student body. So, in my experience, women researchers are more often found working on the user side of MT, while the development side is more dominated by male researchers.
V: I’m on the side of MT development, and the observation here is the same as for computer science in general. There are many more men than women in this domain. I’m not sure that there is a difference between the industry and academia, though it seems that it’s already the case among computer science graduates, who seem to be “equally” (?) distributed between industry and academia.
S: My experience would be similar to Lynne’s. If we’re focusing only on MT “development,” then the majority are male, but I see a growing number of female academics in computer science in general and in the field of natural language processing specifically. However, I think this represents the traditional gender imbalances between science and humanities. If, on the other hand, we broaden out what we mean by “working” with MT, then the picture is more positive. I know of many women working in MT client support, MT evaluation, MT process integration, and MT usage.
J: Maybe I’m barking up the wrong tree here, but don’t you think that academia (i.e., research) and development work very closely together in the case of MT?
S: Yes, that has been my experience, but I wouldn’t say it’s true of all researchers’ experience. The research funding agencies are increasingly requiring collaboration with commercial and nonprofit organizations and are (quite rightly) demanding evidence of “impact” for the “citizens” they represent. We’ll hopefully see more collaboration as a result. The challenge for those of us in translation studies is to ensure that we’re not last-minute add-ons to projects that simply tick a box. My message to MT researchers is that the translation studies and professional translation community is here, and we have a lot to offer and are open for collaboration. Develop with us not for us.
L: Yes, academics and developers do work closely in the case of MT, but at the moment the academics are coming more often from computer science departments than from translation departments. But I agree with Sharon that things are moving in a positive direction. So-called “action” or “participatory” research, which originated in the population and public health domains, is being adopted more widely now and is very relevant to MT research. In a nutshell, participatory research means that researchers are taking steps to include the communities that their work is intended to help more fully in the research process. As Sharon notes, the idea is that the community (e.g., MT users) would not be an afterthought but would be more active participants in the research design process as well. In this way, the research and resulting tools would hopefully better meet the needs of users.
J: So, you don’t think that MT would benefit from proactively working with linguists as part of their development teams? I have admittedly asked developers this question a number of times, both in the areas of statistical and neural machine translation, and have typically received strongly worded answers—which I will not disclose here—but I would be interested in what you have to say about this.
V: That’s an interesting question. I’ve worked with linguists on several occasions during my career (mostly on natural language processing problems other than MT), which has always been an enriching experience for me. The subject of my PhD was to explore whether syntax could improve statistical MT. The problem is that, so far, the impact of linguistic structures has been relatively limited, and outweighed by the gain from “more data” or “bigger model.”
In my understanding, both linguists and algorithms try to discover the regularities (and irregularities) of the language, or several languages at a time in the context of MT applications. Some of those regularities will be consistent across domains and others will change when we switch from one domain to another (e.g., conversational language versus news articles). The algorithms have far more capacities to adapt to the context/domain switch given that those algorithms have access to the relevant data. Therefore, linguists would have a hard time to compete with machines on the tasks where the data is abandoned. However, when we switch to lower resource tasks, including the translation from very low resource languages, or some translation for very specific domains, we would definitely benefit from the linguistic insights, which could guide MT developers in the design of algorithms.
So, to answer your question, yes, I do believe that linguists could help with MT development when the data is limited or sometimes inexistent. If we go beyond MT tasks (which is pretty well defined), in natural language processing in general, I do believe that linguists’ insights are precious in formulating new challenging tasks for natural language processing. And this is how progress is made.
S: The move to data-driven MT seemed to reduce the importance of linguistics—and of linguists. The improvements of MT output, thanks to neural MT, could be seen as limiting the role of linguists even further. However, there is another way of looking at it. To move neural MT output to the next level, the issues that need to be resolved are linguistic issues (e.g., gender in language, style, register, and cohesion, etc.). I think it would be a big mistake for MT developers to assume that this is just a machine learning problem that will be solved by data.
V: I believe a study by Pierre Isabelle, who served as the principal scientist and group leader of the interactive language technologies group at the National Research Council of Canada, and his colleagues is a good illustration of what Sharon is saying.3 This study creates a challenging test set to evaluate the capacities of various MT systems to handle various linguistic phenomena. On the other hand, what this work and a follow-up work4 show is that even if not perfect, current MT systems are making progress in handling those phenomena.
I think it’s not only about the data. The data itself is multidimensional. Various factors are important, such as the amount of data, the quality of data, and diversity of data. But it’s also about the algorithms, which evolve and are able to handle more data and get more out of the data. For example, in 2019, Naver, which was an early pioneer in the use of user-generated content, released an update to Papago, its automated translation app.5 The update allows the user to control the register of produced translation, including rendering English into honorific Korean. There is a combination of data and smarter algorithms behind this feature. So, to a certain extent, some of the problems cited by Sharon could be partially addressed by more/better data and smarter algorithms. But we definitely need more challenging datasets and better evaluation procedures to progress further. Algorithms such as bilingual evaluation understudy (BLEU) scores won’t be able to trace this kind of progress.
J: Is there a threshold for women to get into this field, and what is the path to become part of it?
L: Is the threshold specific to MT, or does it apply more broadly to tech or even science, technology, engineering and mathematics (STEM)? Are there genuinely fewer women in the field, or do women just have a lower visibility? For instance, in academia overall (across all disciplines), there are nearly as many women as men, but when you get to the senior positions (e.g., full professor), the men account for about 70-75% of the posts, and the women just 25-30% (in Canada, anyway).6
So, the imbalance is not in the total number but in the distribution, with men all bunched up in the senior ranks and women all bunched up in the lower ranks. And this is across all disciplines, so not really tech-specific. Generalized explanations that are given are that women are penalized by taking maternity leave, by having or wanting to do more of the caregiving (both for children and elderly parents), or that they are paid significantly less and so don’t feel motivated to work twice as hard. But I’ve seen specific studies on this problem in academia and specific studies7 on gender pay gap issues, but I’ve never seen specific studies on the MT field.
If you want to work in MT development, you need to have some background in coding, which is not typically part of a degree in languages or translation. Maybe it should be! But at present, it’s not the norm. In Canada, this usually means studying computer science, or some other program that has a strong software engineering component. It’s pretty common nowadays for graduate programs in computer science to offer courses in natural language processing, whereas graduate programs in translation offer courses in translation technologies that focus on technology use rather than on its development. So, to my mind, the most typical path to MT development would be through a computer science program rather than through a translation program. There certainly seem to be fewer women in computer science programs at the present time, though there are initiatives to increase the number of women in the STEM fields, so hopefully we’ll see progress in this regard moving forward.
V: As far as I know, it’s generally encouraged to increase diversity in the computer science field in general, and MT development is a subfield of computer science in this context. Both companies and universities strongly encourage qualified women candidates to apply for jobs, and various programs exist to make women’s presence in the field stronger. I don’t think one can define any “threshold,” though as I mentioned previously women are already under-represented in computer science departments.
S: Any imbalance in the field of MT mirrors the imbalance in STEM. In my institution, I see proactive efforts to address this issue—not just in STEM but in computer science and natural language processing as well—by seeking to increase the number of women studying in these fields. In fact, at the ADAPT Centre for Digital Content Technology at Dublin City University, there are currently seven men and 10 women, the latter ranging from the level of professor, through research fellow, post-doc, PhD, and project staff.
There’s another issue here that I think is equally important, which is interdisciplinarity. As Lynne says, the route for anyone into MT development is normally through a computer science (or related) degree. Most MT developers “see” the world through that lens. In MT development and deployment, that typically led to a focus on measuring “success” in computational ways (through BLEU scores, for example), using data that was limited and not checked for quality. Increases in BLEU scores (or equivalent) are emphasized rather than an impact on end users. Having people on your MT development team with linguistic, translation, and human-computer interaction skills in general means that you have a much stronger team who see the world through different lenses. This should ultimately make your science and technology more robust and acceptable. The big issue for me is not how many women are working with MT, but how many people from different disciplines are contributing to the development and measurement of success?
J: What I think I’m hearing all three of you say is that people with linguistic skills would be helpful. And if that’s the case, do they necessarily need the coding and STEM skills you mention? (Although, I really like Lynne’s suggestion that coding should probably be part of translation programs!)
S: Yes, but a considerable challenge is bridging gaps. Therefore, it makes sense for the computational people to understand translation (and translators and end users of translation) and for linguists to understand coding. We’ve started to address this in our MSc in translation technology program, where our students used to take a course in Java programming (now moving to Python). This course is also delivered to students in other faculties, and I don’t mind boasting that our students do really well, probably because they “get” language. This opens the door for them into MT development companies, where they will not necessarily do coding but will understand what’s going on. Other master’s programs have also started to introduce coding modules.
V: I totally agree with Sharon. I think it’s important that we “speak the same language” to better work together.
L: At the moment, our translation program doesn’t include coding, although students could certainly take this as an elective. There are some humanities coding courses offered, such as through the digital humanities minor. But we don’t have a program specifically dedicated to translation technology, as Sharon describes at Dublin City University. For many years, universities were very discipline-based, and so language and computing were essentially in separate silos. But in the past 50 years or so, interdisciplinarity has begun to emerge. In the early days, it was mostly just lip service since the longstanding siloed structures of academia made it difficult to put into action. However, these structural hurdles are gradually disappearing, and I suspect that as universities continue to embrace interdisciplinarity, more programs such as “translation technology” will appear. And it’s in this interdisciplinary space that people can be properly supported to develop hybrid skill sets (e.g., language plus coding) and then go on to act as bridges.
J: Would this field look different if more women were present? Both and maybe separately in academia and industry. Would MT itself be different?
V: I’m not sure how the men/women parity would transform the field of MT development. What would be interesting is if MT developers started to work closely with MT users (who, as Lynne mentioned, are mostly women). That could definitely push MT research in a slightly different direction, which could benefit both sides (MT developers and MT users).
L: Some very interesting recent work on gender bias in MT has been done by Eva Vanmassenhove, who recently completed her PhD on MT at the School of Computing at Dublin City University and who is now an assistant professor at Tilburg University in the Netherlands.8 Vanmassenhove looked at how corpus-based MT systems, such as neural MT systems, can perpetuate and even exaggerate any gender bias found in the training corpora. So, this is a case where a woman MT researcher explored the topic of gender bias, and perhaps that’s a topic that would not necessarily have been investigated as readily by a man. Thanks to this work, Google Translate has undertaken to seek ways to reduce gender bias in their system, which means that the MT output of tomorrow will hopefully look different than the MT output of today. I also agree with Vassilina that finding ways to bring MT developers and those who study user issues in MT together for more conversations is also important. And I think there actually has been improvement in this area in recent years. The MT Summit conferences, as well as some European Association for MT conferences are increasingly offering different “tracks” (e.g., research and development track, user track, etc.), which offer opportunities for people with different interests in MT to come together and share and learn from each other. Let’s hope this trend continues!
S: This is a difficult question to answer without simply speculating, but I find myself echoing the thoughts of Vassilina and Lynne. Much more emphasis has been placed recently on the end user experience of MT. This has actually been driven from the translation side of the house (for obvious reasons), the majority of whom are women. We want to know how good MT is, for what types of text, domains, for which use-case scenarios, and how it impacts a wide variety of users. We are also asking questions about ethics, fair use, etc. (Although this discussion is not only driven by women!) The field is much richer when different types of researchers and users come together. I would like to see more of that!
J: I really want to thank you for this conversation. I learned so much. I also want to thank you all for your work and the thoughtfulness with which you conduct it.
Remember, if you have any ideas and/or suggestions regarding helpful resources or tools you would like to see featured, please e-mail Jost Zetzsche at jzetzsche@internationalwriters.com.
Notes
- Zetzsche, Jost. “Thoughts on Naver Papago with MT Engineer Lucy Park,” The ATA Chronicle (November/December 2019), 30, http://bit.ly/Naver-Papago.
- Microsoft: Global Diversity and Inclusion, http://bit.ly/microsoft-diversity.
- Isabelle, Pierre, Colin Cherry, and George Foster. “A Challenge Set Approach to Evaluating Machine Translation.” In the Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics, 2017), http://bit.ly/Isabelle-Pierre.
- Isabelle, Pierre, and Kuhn, Roland. A Challenge Set for French → English Machine Translation (National Research Council of Canada, 2018), http://bit.ly/Pierre-challenge-MT.
- Jun-sul, Yeo. “Naver Updates Translation Service for Korean Honorifics,” The Korean Herald (January 17, 2019), http://bit.ly/Naver-Korean-honorifics.
- Samson, Natalie, and Anqi Shen. “A History of Canada’s Full-Time Faculty in Six Charts” (University Affairs, 2018), http://bit.ly/Canada-faculty.
- Cummings, Madeleine. “Pay Gap between Male and Female Professors Continues to Plague Canadian Universities,” CBC (September 28, 2020), http://bit.ly/Professors-pay-gap.
- Vanmassenhove, Eva, Christian Hardmeier, and Andy Way. “Getting Gender Right in Neural Machine Translation.” In the Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics, 2018), http://bit.ly/gender-right-NMT.
Jost Zetzsche, CT is chair of ATA’s Translation and Interpreting Resources Committee. He is the author of Translation Matters, a collection of 81 essays about translators and translation technology. Contact: jzetzsche@internationalwriters.com.