Machine Learning
homework代写 | 代做report | Python代写 | Neural Network | Machine Learning代写 | 神经网络 – 本题是一个利用Python进行练习的代做, 对Python的流程进行训练解析, 是比较典型的report/Network/network/Python等代写方向, 这是值得参考的homework代写的题目
My sites/ 21S-STATS413-80/ Homeworks/ homework 03 --- Transformers & GPT Spring^2021 - Week 7
Spring 2021 - STATS413-80 - WU
Machine Learning
Homework 03 — Transformers & GPT
Transformer/BERT/GPT
You also do not need to train the whole model. Use the huggingface transformers and its pretrained model. Test and analysis it.
- Test: Use pretrained BERT, predict the masked token. Use pretrained GPT-2, generate the paragraph. Print the results.
- Analsis: Read the source code (will help you find where it is later). According to that, write the network structure and key equations.
Detailed requirement
1. As usual, write introduction, key equations, and network structures.
BERT:
Embedding: what it included
Transformers: what is self-attention, how is the linear layer likes.
Output: How the output of transformers become words.
GPT: How to predict next word based on previous words.
For both method, read the source code and find the corresponding line for each components. (e.g. self-attention is on line xxx,
embedding is on line xxx). VS code with Python extention is strongly recommended to read these code.
BERT: Source code
GPT: Source code
2. BERT: Mask token prediction:
Follow the given notebook, change the following:
1. Change the sentence to your own one. Change 3 different sentences.
2. Change bert-base to other 2 models, refer to Model Zoo. You can use other language if you prefer, French or Chinese. Remember
to change the corresponding class if you do not use bert. Compare 3 results.
3.
3. GPT: Sentence prediction:
Follow the given notebook, change the following:
1. Change the sentence to your own one. Change 3 different sentences.
2. Try to generate the whole paragraph. That is, more than 200 words.
3. Also, you may change models and see how it is going.
Grading (total 100pts)
Homework report (90pts)
Get the mobile app
example_transformers.ipynb 8 May 2021, 12:29 AM PDT
BERT (45pts)
introduction: 5pts
main equation, structures: 15pts
Find components in the source code: 15pts
Experiments: 10pts
GPT(45pts)
introduction: 5pts
main equation, structures: 15pts
Find components in the source code: 15pts
Experiments: 10pts
Submit the code (py or ipynb): 10pts
1.
Submission status
Submission status No attempt
Grading status Not graded
Due date Friday, 14 May 2021, 11:59 PM PDT
Time remaining 4 days 3 hours
Last modified -
Submission
comments Comments (0)
Add submission
You have not made a submission yet.
Homework 02 --- Resnet & ...
Jump to...
2021 UC Regents Contact About Privacy Copyright UCLA links | UCLA Registrar MyUCLA Accessibility Couns/PsychSvc (CAPS)