The need for summarizing long medical scan videos for automatic triage in Emergency Departments and transmission of the summarized videos for telemedicine has gained significance during the COVID-19 pandemic. However, supervised learning schemes for summarizing videos are infeasible as manual labeling of scans for large datasets is impractical by frontline clinicians. This work presents a methodology to summarize ultrasound videos using completely unsupervised learning schemes and is validated on Lung Ultrasound videos. A Convolutional Autoencoder and a Transformer decoder is trained in an unsupervised reinforcement learning setup i.e., without supervised labels in the whole workflow. Novel precision and recall computation for ultrasound videos is also presented employing which high Precision and F 1 scores of 64.36\% and 35.87\% with an average video compression rate of 78\% is obtained when validated against clinically annotated cases. Even though demonstrated using lung ultrasound videos, our approach can be readily extended to other imaging modalities.