Zeeshan 02-19-2020

Internship at Amazon and forthcoming thesis

What is transfer learning?

• Transfer learning is a a variety of different things. For a taxonomy read Ruder 2019.
• pretraining of word embeddings is probably the most famous form of transfer learning.

Multi-task learning

Hard vs soft parameter sharing Hard parameter sharing literally shares some of the initial layers and then has task specific layers towards the end.

Soft parameter sharing uses soem method of regularization to force common layers for the two tasks to be close to eachother.