Use bigger datasets for CNN in hope of better performance. A new data set for sports video classification: sports-1M.
CNN in one frame is about the same as many frames. CNN is good at image but not modeling temporal sequences. The result is not good.
transfer learning for UCF-100 dataset.
transfer learning code : dhttp://www.cse.ust.hk/TL/index.html