| 澶辨晥閾炬帴澶勭悊 |
|
澶фā鍨嬶紙LLMs錛夎緇冮泦闈?PDF 涓嬭澆
鐩稿叧鎴浘錛?/strong>
 涓昏鍐呭錛?/strong>
1. SFT錛堟湁鐩戠潱寰皟錛夌殑鏁版嵁闆嗘牸寮忥紵
涓€闂竴絳?/span>
2. RM錛堝鍔辨ā鍨嬶級鐨勬暟鎹牸寮忥紵
涓€涓棶棰?/span> + 涓€鏉″ソ鍥炵瓟鏍蜂緥 + 涓€鏉″樊鍥炵瓟鏍蜂緥
3. PPO錛堝己鍖栧涔狅級鐨勬暟鎹牸寮忥紵
鐞嗚涓婃潵璇達紝涓嶉渶瑕佹柊澧炴暟鎹€傞渶瑕佹彁渚涗竴浜?/span>prompt錛屽彲浠ョ洿鎺ョ敤sft闃舵鐨勯棶銆傚彟澶栵紝闇€瑕侀檺鍒?/span>
妯″瀷涓嶈鍋忕鍘熸ā鍨嬪お榪滐紙ptx loss錛夛紝涔熷彲浠ョ洿鎺ョ敤sft鐨勬暟鎹€?/span>
4. 鎵炬暟鎹泦鍝噷鎵撅紵
鎺ㄨ崘Alpaca-COT錛屾暟鎹泦鏁寸悊鐨勯潪甯稿叏錛岀溂鑺辯輯涔便€?/span>
5. 寰皟闇€瑕佸灝戞潯鏁版嵁錛?/strong>
鍙栧喅浜庨璁粌鏁版嵁鍜屽井璋冧換鍔＄殑鏁版嵁鍒嗗竷鏄惁涓€鑷達紝鍒嗗竷涓€鑷達紝100鏉″氨澶燂紝鍒嗗竷宸紓澶у氨闇€瑕佸
浜涙暟鎹紝鍗冩潯鎴栬€呬竾鏉′互涓婁負浣熾€?/span>
鑷繁鐨勪換鍔″鏉傛垨鑰呬笅娓鎬換鍔¤涓氭瘮杈冨喎闂紝濡傝嵂鍝佸悕縐拌瘑鍒換鍔★紝鍒欓渶瑕佽緝澶氱洃鐫ｆ暟鎹€傝繕鏈夊井
璋冨ぇ妯″瀷鏃訛紝涓€閬嶆槸璁頒笉浣忕殑銆?/span>100鏉＄殑寰皟鏁版嵁錛?/span>epochs=20鎵嶈兘紼沖畾鎷熷悎浠誨姟瑕佹眰銆?/span>
6. 鏈夊摢浜涘ぇ妯″瀷鐨勮緇冮泦錛?/strong>
棰勮緇冩暟鎹泦togethercomputer/RedPajama-Data-1T銆岀孩鐫¤。銆嶅紑婧愯鍒掓€誨叡鍖呮嫭涓夐儴鍒嗭細
棰勮緇冩暟鎹泦RedPajama-Data-1T宸插紑婧愶紝鍖呮嫭涓冧釜瀛愰泦錛岀粡榪囬澶勭悊鍚庡緱鍒扮殑token鏁伴噺澶ц嚧鍙?/span>
浠ュ尮閰?/span>Meta鍦ㄥ師濮?/span>LLaMA璁烘枃涓姤鍛婄殑鏁伴噺錛屽茍涓旀暟鎹澶勭悊鐩稿叧鑴氭湰涔熷凡寮€婧愩€?/span>
瀹屾暣鐨?/span>RedPajama-Data-1T鏁版嵁闆嗛渶瑕佺殑瀛樺偍瀹歸噺涓哄帇緙╁悗3TB錛岃В鍘嬪悗5TB銆?/span>
CoT寰皟鏁版嵁闆嗭細Alpaca-CoT 閲岄潰鍖呮嫭甯哥敤鐨?/span>alpaca錛?/span>CoT絳夋暟鎹泦錛屾湁涓枃鐨勩€?/span>
7. 榪涜棰嗗煙澶фā鍨嬮璁粌搴旂敤鍝簺鏁版嵁闆嗘瘮杈冨ソ錛?/strong>
閫氳繃鍒嗘瀽鍙戠幇鐜版湁鐨勫紑婧愬ぇ妯″瀷榪涜棰勮緇冪殑榪囩▼涓細鍔犲叆鏁版嵁銆佽鏂囩瓑鏁版嵁銆備富瑕佹槸鍥犱負榪欎簺鏁?/span>
鎹殑鏁版嵁璐ㄩ噺杈冮珮錛岄鍩熺浉鍏蟲€ф瘮杈冨己錛岀煡璇嗚鐩栫巼錛堝瘑搴︼級杈冨ぇ錛屽彲浠ヨ妯″瀷鏇撮€傚簲鑰冭瘯銆傜粰鎴?/span>
• 楂樿川閲忋€佸ぇ瑙勬ā銆侀珮瑕嗙洊搴︾殑棰勮緇冩暟鎹泦錛?/span>
• 鍦ㄩ璁粌鏁版嵁闆嗕笂璁粌鍑虹殑鍩虹妯″瀷錛?/span>
• 鎸囦護璋冧紭鏁版嵁闆嗗拰妯″瀷錛屾瘮鍩烘湰妯″瀷鏇村畨鍏ㄣ€佸彲闈犮€?/span>
鎵爜鍔?/span>
鏌ョ湅鏇村浠嚜宸辮繘琛屽ぇ妯″瀷棰勮緇冪殑鏃跺€欐彁渚涗簡涓€涓弬鑰冦€傚悓鏃墮鍩熺浉鍏崇殑緗戠珯鍐呭銆佹柊闂誨唴瀹逛篃鏄瘮杈冮噸
瑕佺殑鏁版嵁銆?/span>
|
| Java1234瀹樻柟緹?5錛?/td> | 
|
| Java1234瀹樻柟緹?5錛?/td> | 838462530 |
鑻忓叕緗戝畨澶?32061202001004鍙?/p>
