In image super-resolution (SR), the main aim is to improve the quantitative results, such as PSNR and perceptual quality of the image. Some models can improve one or both of them such as the Transformer model and diffusion models. In this paper, we take the merit of both the Transformer and diffusion models to improve the real-world image SR model. So, we designed a fusion strategy for the outcome images of both the Transformer and diffusion. This fusion represents the integration of multiple sources of information or features, creating a more cohesive and comprehensive representation. A new model called the double diffusion image super-resolution (DDiffSR) model is proposed based on fusing both the Transformer and the diffusion model. The transformer model is used to extract a compact image SR prior. Meanwhile, the diffusion model is based on the residual denoise diffusion model to generate high-resolution images. The new DDiffSR model achieved state-of-the-art traditional and real-world image SR results. Also, the model achieved more appealing visual results. For example, our model enhanced the PSNR by 1.85 dB for the Set14 dataset.