Influence function

http://arxiv.org/abs/2402.04333

1. Purpose

   Given just a a handful of examples embodying a specific capability, how can we effectively select relevant fine-tuning data from a large collection of instruction datasets?

2. Influence function with Adam(LESS)

   • Tayor expansion

     $$\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^{t+1})\approx \ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^t)+⟨\mathrm{\nabla }\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^t),{\mathit{\theta }}^{t+1}-{\mathit{\theta }}^t⟩$$$${\mathit{\theta }}^{t+1}-{\mathit{\theta }}^t=-{\eta }_t\mathrm{\nabla }\ell (\mathit{z};{\mathit{\theta }}^t)$$$$\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^{t+1})-\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^t)\approx -{\eta }_t⟨\mathrm{\nabla }\ell (\mathit{z};{\mathit{\theta }}^t),\mathrm{\nabla }\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}^t)⟩$$

   • Trajectory influence

     $${\mathrm{Inf}}_{\mathrm{SGD}}(\mathit{z},{\mathit{z}}^{\prime })\triangleq \sum _{i=1}^N{\overline{\eta }}_i⟨\mathrm{\nabla }\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}_i),\mathrm{\nabla }\ell (\mathit{z};{\mathit{\theta }}_i)⟩$$

   • Extension to Adam

     $$\begin{array}{c}{\mathit{\theta }}^{t+1}-{\mathit{\theta }}^t=-{\eta }_t\mathrm{\Gamma }(\mathit{z},{\mathit{\theta }}^t)\\ \mathrm{\Gamma }(\mathit{z},{\mathit{\theta }}^t)\triangleq \frac{{\mathit{m}}^{t+1}}{\sqrt{{\mathit{v}}^{t+1}+\mathit{ϵ}}}\\ {\mathit{m}}^{t+1}=({\beta }_1{\mathit{m}}^t+(1-{\beta }_1)\mathrm{\nabla }\ell (\mathit{z};{\mathit{\theta }}^t))/(1-{\beta }_1^t)\\ {\mathit{v}}^{t+1}=({\beta }_2{\mathit{v}}^t+(1-{\beta }_2)\mathrm{\nabla }\ell {(\mathit{z};{\mathit{\theta }}^t)}^2)/(1-{\beta }_2^t)\end{array}$$

   • Adam Influence

     $${\mathrm{Inf}}_{\text{Adam }}(\mathit{z},{\mathit{z}}^{\prime })\triangleq \sum _{i=1}^N{\overline{\eta }}_i\frac{⟨\mathrm{\nabla }\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}_i),\mathrm{\Gamma }(\mathit{z},{\mathit{\theta }}_i)⟩}{\Vert \mathrm{\nabla }\ell ({\mathit{z}}^{\prime };{\mathit{\theta }}_i)\Vert \Vert \mathrm{\Gamma }(\mathit{z},{\mathit{\theta }}_i)\Vert }$$

3. Problems&Improvements

   • Random Projection (JL lemma)

     

     

   • Lora