In this study, new tetrahydroisoquinoline compounds were synthesized by reaction of 7-Acetyl-4-cyano-1,6-dimethyl-6-hydroxy-8- (3-nitrophenyl or 4-nitrophenyl)-5,6,7,8-tetrahydrosoquinoline-3(2H)-thiones with methyl iodide, chloro acetonitrile, ethyl chloroacetate to produce compounds 3–5 and reacted with N-arylchloroacetamides reagents to gave tetrahydroisoquinolin-3-ylthio) acetamides compounds 6a–c, 8a–b which can cyclized to 6,7,8,9-tetrahydrothieno[2,3-c]Isoquinoline-2-carboxamides compounds 7a–c, 9a–b. Also react with N-(benzthiazol-2-yl)-2-chloroacetamideto give compound 10. The structures of all newly synthesized compounds were characterized by elemental and spectral analyses. Also, most of the synthesized compounds were evaluated for their anticancer activities aganist MCF7 and HEPG2 cell lines. From the result we found that the most active compound against the MCF7 cell lines was compound 8b, and the most active compound against HEPG2 cell lines was compound 3. Then the effects of compound 3 on the HEPG2 cell line was investigated using an apoptotic Annexin V-FITC test and flow cytometry. Compound 3 induced a 59-fold increase in HEPG2 cell line apoptosis and cell cycle arrested at the G0-G1, G2/M phases. Moreover, the molecular docking study was applied and the result showed that compounds 8b bind to the RET enzyme with binding energies of − 6.8 kcal/mol in comparison with standard alectinib, which exhibits a binding energy of − 7.2 kcal/mol. Compound 3 can bind with HSP 90 with a binding energy (ΔG) of − 6.8 kcal/mol, which was comparable to the standard Onalespib (− 7.1 kcal/mol).