Resources >‏ In the Media >‏ Latest News Details

Latest News Details

Reading a neural network’s mind

Publication Date:
12/12/2017
Category:
In the Media
Neural networks MIT.jpg
Neural networks, which learn to perform computational tasks by analyzing huge sets of training data, have been responsible for the most impressive recent advances in artificial intelligence, including speech-recognition and automatic-translation systems.

During training, however, a neural net continually adjusts its internal settings in ways that even its creators can’t interpret. Much recent work in computer science has focused on clever techniques for determining just how neural nets do what they do.

In several recent papers, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Qatar Computing Research Institute have used a recently developed interpretive technique, which had been applied in other areas, to analyze neural networks trained to do machine translation and speech recognition.

They find empirical support for some common intuitions about how the networks probably work. For example, the systems seem to concentrate on lower-level tasks, such as sound recognition or part-of-speech recognition, before moving on to higher-level tasks, such as transcription or semantic interpretation.

But the researchers also find a surprising omission in the type of data the translation network considers, and they show that correcting that omission improves the network’s performance. The improvement is modest, but it points toward the possibility that analysis of neural networks could help improve the accuracy of artificial intelligence systems.

“In machine translation, historically, there was sort of a pyramid with different layers,” says Jim Glass, a CSAIL senior research scientist who worked on the project with Yonatan Belinkov, an MIT graduate student in electrical engineering and computer science. “At the lowest level there was the word, the surface forms, and the top of the pyramid was some kind of interlingual representation, and you’d have different layers where you were doing syntax, semantics. This was a very abstract notion, but the idea was the higher up you went in the pyramid, the easier it would be to translate to a new language, and then you’d go down again. So part of what Yonatan is doing is trying to figure out what aspects of this notion are being encoded in the network.”

The work on machine translation was presented recently in two papers at the International Joint Conference on Natural Language Processing. On one, Belinkov is first author, and Glass is senior author, and on the other, Belinkov is a co-author. On both, they’re joined by researchers from the Qatar Computing Research Institute (QCRI), including Lluís Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and Stephan Vogel. Belinkov and Glass are sole authors on the paper analyzing speech recognition systems, which Belinkov presented at the Neural Information Processing Symposium last week.

To read more click here.



Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

Forbes fake news pic.jpg

Can AI Put An End To Fake News? Don't Be So Sure

07/10/2018

Fake news was the Collin’s word of the year for 2017 with good reason. In a year where politics-as-usual was torn apart at the seams, high-profile scandals rocked our faith in humanity and the ...

Read More

roadtracer.png

MIT/QCRI system uses machine learning to build road maps

22/04/2018

Map apps may have changed our world, but they still haven’t mapped all of it yet. Specifically, mapping roads can be difficult and tedious: even after taking aerial images, companies still have to ...

Read More

Economist story pic.JPG

Improving disaster response efforts through data

08/02/2018

Extreme weather events put the most vulnerable communities at high risk. How can data analytics strengthen early warning systems and and support relief efforts for communities in need? The size and ...

Read More

Events

2019

Dr Farnam Jahanian (2).jpg

“The Future of Higher Education in the Age of Technological Disruption” by CMU President Dr. Farnam Jahanian

Download ICS File 24/03/2019 ,

Dr. Farnam Jahanian, the President of Carnegie Mellon University, will deliver a public lecture, “The Future of Higher Education in the Age of Technological Disruption” at CMU’s Qatar campus on Sunday, March 24.

Read More

QCRI CSAIL Logos.JPG

QCRI - MIT CSAIL 2019 Annual Project Review

Download ICS File 25/03/2019 ,

Executive Overview Sessions Open to publi Date: March 25, 2019 Time: 10:15AM - 5:15PM Venue: Hamad Bin Khalia Reseach Complex Multipurpose Room To view agenda, please click here . To RSVP to this ...

Read More

Torralba.png

"Learning to See" Public talk by Professor Antonio Torralba (MIT-CSAIL)

Download ICS File 25/03/2019 ,

Visit by Antonio Torralba, who teaches machines to automate tasks that a human visual system can accomplish, is part of annual spring research update between QCRI and MIT-CSAIL.

Read More

News

QCRI-iMMAP MOU.jpg

QCRI and iMMAP announce Memorandum of Understanding

03/03/2019

Pact aims to apply data analysis and artificial intelligence techniques to solve humanitarian problems.

Read More

UNDP workshop.JPG

UNDP partners with QCRI to use AI for social good

11/02/2019

Qatar forum on leveraging AI to solve humanitarian problems fills to capacity.

Read More

C. Mohan pic.jpg

Renowned computing expert C. Mohan to bust blockchain myths in Qatar talk

22/01/2019

Well-known inventor of database recovery algorithms to deliver keynote at QCRI's first blockchain workshop.

Read More