Inferring transient molecular structural dynamics from diffraction data is an ambiguous task that often requires different approximation methods. In this paper, we present an attempt to tackle this problem using machine learning. Although most recent applications of machine learning for the analysis of diffraction images apply only a single neural network to an experimental dataset and train it on the task of prediction, our approach utilizes an additional generator network trained on both synthetic and experimental data. Our network converts experimental data into idealized diffraction patterns from which information is extracted via a convolutional neural network trained on synthetic data only. We validate this approach on ultrafast electron diffraction data of bismuth samples undergoing thermalization upon excitation via 800 nm laser pulses. The network was able to predict transient temperatures with a deviation of less than 6% from analytically estimated values. Notably, this performance was achieved on a dataset of 408 images only. We believe that employing this network in experimental settings where high volumes of visual data are collected, such as beam lines, could provide insights into the structural dynamics of different samples.