万文长字总结「类别增量学习」的前世今生、开源工具包（3）

六、PyCIL: A Python Toolbox for Class-Incremental Learning

我们开源了一个基于pytorch的Class-IL框架：PyCIL。其中不仅包含了如EWC、iCaRL等一批早期的fundamental的方法，还包含了一些现在state-of-the-art的Class-IL算法，希望能够帮助一些希望了解和研究相关领域的学者。项目地址：PyCIL^[13]

Methods Reproduced

FineTune: Baseline method which simply updates parameters on new task, suffering from Catastrophic Forgetting. By default, weights corresponding to the outputs of previous classes are not updated.

• EWC: Overcoming catastrophic forgetting in neural networks. PNAS2017 [paper]
  
• LwF: Learning without Forgetting. ECCV2016 [paper]
  
• Replay: Baseline method with exemplars.
  
• GEM: Gradient Episodic Memory for Continual Learning. NIPS2017 [paper]
  
• iCaRL: Incremental Classifier and Representation Learning. CVPR2017 [paper]
  
• BiC: Large Scale Incremental Learning. CVPR2019 [paper]
  
• WA: Maintaining Discrimination and Fairness in Class Incremental Learning. CVPR2020 [paper]
  
• PODNet: PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning. ECCV2020 [paper]
  
• DER: DER: Dynamically Expandable Representation for Class Incremental Learning. CVPR2021 [paper]
  
• Coil: Co-Transport for Class-Incremental Learning. ACM MM2021 [paper]
  

部分实验结果

实验结果（一） 实验结果（二）

参考

1. Zhiyuan Chen; Bing Liu; Ronald Brachman; Peter Stone; Francesca Rossi, Lifelong Machine Learning: Second Edition , Morgan & Claypool, 2018. https://ieeexplore.ieee.org/document/8438617
   
2. Catastrophic forgetting in connectionist networks https://www.sciencedirect.com/science/article/pii/S1364661399012942
   
3. PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning https://arxiv.org/abs/1711.05769
   
4. Three scenarios for continual learning https://arxiv.org/abs/1904.07734
   
5. Learning without Forgetting https://arxiv.org/abs/1606.09282
   
6. Distilling the Knowledge in a Neural Network https://arxiv.org/abs/1503.02531
   
7. iCaRL: Incremental Classifier and Representation Learning https://arxiv.org/abs/1611.07725
   
8. Large Scale Incremental Learning https://arxiv.org/abs/1905.13260
   
9. Maintaining Discrimination and Fairness in Class Incremental Learning https://arxiv.org/abs/1911.07053
   
10. DER: Dynamically Expandable Representation for Class Incremental Learning https://arxiv.org/abs/2103.16788
    
11. https://arxiv.org/abs/1801.01423https://arxiv.org/abs/1801.01423
    
12. Co-Transport for Class-Incremental Learning https://arxiv.org/abs/2107.12654
    
13. PyCIL: A Python Toolbox for Class-Incremental Learning https://arxiv.org/abs/2112.12533