The evolving field of computational image analysis has its applications in the industry, manufacturing and biological sciences, especially in the field of medical imaging. Medical imaging and computational physics have evolved together during the past decades with the advancement in the field of artificial intelligence (AI). Deep learning is the sub-domain of AI that mostly deals with imaging data for classification, segmentation and reconstruction. The time series of medical images of different patients, with different staging are categorized based on the physical and biological consequences. The hypothesis of the current research is that the deep learning tool, if trained on several patients, can identify the stage of cancer swiftly for fresh data sets. During this research, an advance Convolutional Neural Network (CNN) strategy is adopted to classify the cancer stage for a group of patients of gastric cancer. The CNN model makes use of skipping connections for better prediction. CNNs have been quite popular in medical imaging for their ability of feature detection. CNNs are used in the recent literature for the analysis of images. During this research, we have used the state-of-the-art Matlab ResNet CNN toolbox for the analysis of the images obtained from esophageal and gastric cancer patients. It was concluded that RESNET50 is a reliable algorithm for the determination of tumor mass on CT Scans. Moreover, the performance of the model can be improved by giving a comparatively larger data set as an input to the model. Inspired from Caltech101, a logic related to RESNET50 was adopted. The data was processed and an algorithm was designed to develop a mapping, based on the mass of tumor. The algorithm designed successfully identified the images, randomly picked from different patients, based on the image features.