Scribd
Upload
66 views14 pages

Neural Network and Deep Learning

This document discusses the evolution of machine translation, from early rule-based systems to modern neural network approaches. It describes how statistical machine translation systems trained on large datasets emerged in the 1970s and benefited from parallel text corpora. Recently, deep learning techniques have been widely applied to machine translation, using neural networks with multiple hidden layers to learn complex representations from data. Many current works focus on applying these neural models to translation and language modeling tasks, achieving significant improvements over prior statistical approaches. The document concludes that neural methods appear poised to further transform the field of machine translation.

Uploaded by

MAYS Shaker
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
66 views14 pages

Neural Network and Deep Learning

This document discusses the evolution of machine translation, from early rule-based systems to modern neural network approaches. It describes how statistical machine translation systems trained on large datasets emerged in the 1970s and benefited from parallel text corpora. Recently, deep learning techniques have been widely applied to machine translation, using neural networks with multiple hidden layers to learn complex representations from data. Many current works focus on applying these neural models to translation and language modeling tasks, achieving significant improvements over prior statistical approaches. The document concludes that neural methods appear poised to further transform the field of machine translation.

Uploaded by

MAYS Shaker
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd

Mayes F.

Shaker

Neural Network And


Deep Learning
Introduction

• Machine translation (MT ) has been approached mainly


following a rule-based or corpus-based strategy.
• Rulebased MT systems date back early 70s with
• The idea behind rule-based approaches is that
transformation from source to target is done by means of
performing an analysis of the source text, transferring this
new source representation to a target representation and
generating the final target text.
statistical approach
• statistical approach benefits from being
trained on large datasets. Normally,
statistical MT uses parallel texts at the level
of sentences, it uses co-occurrences to
extract a bilingual dictionary, and finally, it
uses monolingual text to compute a
language model which estimates the most
fluent translation text in the target language.
The Big Turn

New training and optimization algorithms for neural


networks, i.e. deep learning techniques
• Neural networks can be defined as a type of statistical learning
algorithms used to estimate functions that can have a large
number of inputs.
• Neural networks are organised in layers, including an input and an
output layers and, in between, considering one or several hidden
layers. Each layer is composed by neurons which is the elementary
unit of the network.
• Each neuron receives one or several inputs for which the neuron
performs a weighted sum of the inputs and pass it through a non-
linear function (activation) to produce the output.
• Deep learning is about learning representations with
multiple levels of abstraction and complexity
• There has been a lot of recent activity from the
scientific community in using deep learning in MT
refelected in, for example, an explosion in the number
of works in relevant conferences from 2014 up to date.
Most works are applied
to main translation and
language models
Conclusion
• It is adventurous how neural algorithms are going to
impact MT in the future but it seems that they are here to
stay as proven by recent news on big companies adopting
the neural MT approach e.g. Google and Systran.
• Deep learning is already taking the field dramatically
further as shown by the appearence of first end-to-end
speech-to-text translation and multimodal MT and
unsupervised MT
Conclusion
• In the evolution of MT, we have lost rules
(in the transition from the rule to the
statistical-based approach)
• we have lost translation units (in the
transition from the statistical to the
neural-based approach)
For further information,
please feel free to
contact
07726600657

[Link]@[Link]

You might also like