澶辨晥閾炬帴澶勭悊 |
鍩轟簬lora鐨刲lama2浜屾棰勮緇?nbsp; PDF 涓嬭澆
杞澆鑷細(xì)http://www.python222.com/article/1202
鐩稿叧鎴浘錛?/strong>
![]() 涓昏鍐呭錛?/strong>
涓€銆佷負(fù)浠€涔堥渶瑕?瀵?/strong> llama2 鍋?鍩轟簬lora鐨勪簩嬈¢璁粌?
鍔犲叆涓枃璁粌璇枡榪涜llama2鐨勪簩嬈¢璁粌錛岃繖鏍鋒ā鍨嬪氨鍙互澧炲姞鏀寔涓枃杈撳嚭鐨勮兘鍔涖€?/span>
浜屻€佸熀浜?/strong>lora鐨?/strong>llama2浜屾棰勮緇?鐨勭洰鏍囨槸浠€涔堬紵
鍦ㄤ繚鎸侀璁粌妯″瀷鏉冮噸涓嶅彉鐨勬儏鍐典笅錛岄€氳繃娣誨姞棰濆鐨勭綉緇滃眰騫朵粎璁粌榪欎簺鏂板鐨勭綉緇滃眰鍙傛暟錛屽疄鐜板ぇ妯″瀷鐨勯珮
鏁堝井璋冿紙peft錛夈€?/span>
涓夈€佸熀浜?/strong>lora鐨?/strong>llama2浜屾棰勮緇?鐨勬€濇兂鏄粈涔堬紵
鎬濇兂錛氬熀浜庡妯″瀷鏈緛緇村害錛?/span>intrinsic dimension錛夌殑鐞嗚В銆?/span>
“鏈緛緇村害”鏄寚妯″瀷涓湡姝f湁鐢ㄧ殑銆佽兘澶熷獎(jiǎng)鍝嶆ā鍨嬭緭鍑虹殑鍙傛暟鏁伴噺銆?/span>
Aghajanyan鐮旂┒鍙戠幇錛岄璁粌妯″瀷鐨勫唴鍦ㄧ淮搴﹀疄闄呬笂闈炲父灝忥紝鍗沖彧鏈変竴灝忛儴鍒嗗弬鏁板妯″瀷杈撳嚭鏈夋樉钁楀獎(jiǎng)鍝嶃€傚氨鏄?/span>
瀛樺湪涓€涓瀬浣庣淮搴︾殑鍙傛暟錛屽井璋冨畠鍜屽湪鍏ㄥ弬鏁扮┖闂翠腑寰皟鑳借搗鍒扮浉鍚岀殑鏁堟灉
LORA鍋囪妯″瀷鍦ㄤ換鍔¢€傞厤榪囩▼涓潈閲嶇殑鏀瑰彉閲忔槸浣庣З錛?/span>low rank錛?/span>
W=W0+ΔW錛?/span>ΔW=BA
鍙傛暟鏇存柊鑼冨洿錛氬彧璁粌鏂板鐨勭綉緇滃眰鍙傛暟
|