prikaz prve stranice dokumenta Transformer neuronske mreže za obradu prirodnog jezika
  Download
PDF 1.98 MB
master's thesis
Transformer neuronske mreže za obradu prirodnog jezika
2022. urn:nbn:hr:122:471182
Petrović, Darko
University North
University centre Varaždin
Department of Multimedia, Design and Application

Cite this document

Petrović, D. (2022). Transformer neuronske mreže za obradu prirodnog jezika (Master's thesis). Koprivnica: University North. Retrieved from https://urn.nsk.hr/urn:nbn:hr:122:471182

Petrović, Darko. "Transformer neuronske mreže za obradu prirodnog jezika." Master's thesis, University North, 2022. https://urn.nsk.hr/urn:nbn:hr:122:471182

Petrović, Darko. "Transformer neuronske mreže za obradu prirodnog jezika." Master's thesis, University North, 2022. https://urn.nsk.hr/urn:nbn:hr:122:471182

Petrović, D. (2022). 'Transformer neuronske mreže za obradu prirodnog jezika', Master's thesis, University North, accessed 24 February 2025, https://urn.nsk.hr/urn:nbn:hr:122:471182

Petrović D. Transformer neuronske mreže za obradu prirodnog jezika [Master's thesis]. Koprivnica: University North; 2022 [cited 2025 February 24] Available at: https://urn.nsk.hr/urn:nbn:hr:122:471182

D. Petrović, "Transformer neuronske mreže za obradu prirodnog jezika", Master's thesis, University North, Koprivnica, 2022. Available at: https://urn.nsk.hr/urn:nbn:hr:122:471182

Please login to the repository to save this object to your list.
Metadata
TitleTransformer neuronske mreže za obradu prirodnog jezika
Title (english)Transformer neural networks for natural language processing
AuthorDarko Petrović
MentorEmil Dumić (mentor)
Committee memberMario Periša (predsjednik povjerenstva)
Committee memberRobert Geček (član povjerenstva)
Committee memberEmil Dumić (član povjerenstva)
GranterUniversity North
(University centre Varaždin)
(Department of Multimedia, Design and Application)
Koprivnica
Defense date and country2022-09-29, Croatia
Scientific / art field,
discipline and subdiscipline		TECHNICAL SCIENCES
Graphic Technology
Processes of Graphic Reproduction
Abstract
U ovom radu bit će ispitano nekoliko tipova transformer neuronskih mreža (Neural Networks, NN) za obradu prirodnog jezika. U radu će biti opisano područje obrade prirodnog jezika (Natural Language Processing, NLP) koja predstavlja vezu između računala i ljudskih jezika. Bit će objašnjeni i povezani pojmovi, shvaćanje prirodnog jezika (Natural Language Understanding) i stvaranje prirodnog jezika (Natural Language Generation, NLG). Najčešće se za obradu prirodnog jezika koriste različite metode... More strojnog učenja. Posebno će se opisati različiti tipovi dubokih neuronskih mreža koji mogu biti iskorišteni i za obradu prirodnog jezika: RNN (Recurrent Neural Network), ograđena RNN (gated RNN), LSTM (Long Short Term Memory) te transformeri. Zatim će se opisati modeli neuronskog strojnog prevođenja bazirani na pozornosti (attention-based neural machine translation). Potom će se detaljnije opisati transformer tip neuronske mreže, također baziran na pozornosti, te nekoliko tipova takvih istreniranih neuronskih mreža: BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT pretraining approach), ALBERT (A Lite BERT) i XLNet. Usporedbu različitih metoda za strojni prijevod jednog jezika u drugi možemo napraviti koristeći BLEU ocjenu (Bilingual evaluation understudy). U praktičnom dijelu rada bit će prikazano nekoliko tipova transformera i njihovi rezultati za područje obrade prirodnog jezika. Za usporedbu će biti korištena neka od postojećih baza poput skup podataka knjiga Opus (eng. Opus books dataset) za prijevod, koristeći engleski i francuski jezik. Usporedba različitih tipova neuronskih mreža će biti dana koristeći BLEU ocjenu. Less
Abstract (english)
In this paper, several types of transformer neural networks (NN) for natural language processing will be examined. The paper will describe the field of natural language processing (NLP), which represents the connection between computers and human languages. Related concepts, Natural Language Understanding and Natural Language Generation (NLG) will be explained. Different machine learning methods are most often used for natural language processing. Different types of deep neural networks that... More can be used for natural language processing will be described in particular: RNN (Recurrent Neural Network), fenced RNN (gated RNN), LSTM (Long Short Term Memory) and transformers. Then attention-based neural machine translation models will be described. Then the transformer type of neural network, also based on attention, and several types of such trained neural networks will be described in more detail: BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT pretraining approach), ALBERT (A Lite BERT) and XLNet. A comparison of different methods for machine translation from one language to another can be made using the BLEU evaluation (Bilingual evaluation understudy). In the practical part of the work, several types of transformers and their results for the field of natural language processing will be presented. For comparison, some of the existing databases, such as the Opus books dataset, will be used for translation, using English and French. A comparison of different types of neural networks will be given using the BLEU score. Less
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:122:471182
Study programmeTitle: Multimedia Science
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra inženjer/inženjerka grafičke tehnologije (magistar/magistra inženjer/inženjerka grafičke tehnologije)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Terms of useLicence In copyright icon
Created on2023-01-27 12:55:51
accessibility

closeAccessibilityrefresh

If you wish to permanently save changes, click on Save, if not - your settings will be reset when you restart the browser.