It's hard to pinpoint my EXACT first experience with not-Goosebumps horror, but I believe it may have been Clock Tower- a 1995 game that is, incidentally, known as the pioneer of the survival horror genre. It's pixel art, so no gore, and there's no sexual shock value, either.
If you think you can handle something a tiny bit gorier, I recommend Witch's House. The horror genre had (probably still does) a super neat community that made their own games with a tool called RPG Maker. Witch's House began as one of these horror RPG Maker games! A bigger publisher helped them remake and polish it sometime within the last five years. If you don't think you can do gore at all, find the original- there's blood and violence but it cuts out before you see anything gorey. If you think you can handle a little, the recent version has some gorgeous pixel art to gawk at.
I will slap a content warning on this one though for an ending you may not enjoy. If you don't do endings that aren't happy, don't look at this one.
Both of these games involve teenager girls dungeon crawling their way through spooky old houses by solving weird logic puzzles. In Clock Tower, there's multiple endings depending on a number of factors, and you'll get one half of the story for two of the different routes you can take!
In Witch's House there is only one true ending that you do have to do something special to unlock. I... admit I forgot what it is, though.
Watching a youtube playthrough is a perfectly acceptable decision, too. In fact I recommend that. If something is too much for you, you can tap out, then.
Horror is a deeply fascinating genre to me, a very unique way to explore the human condition, and if you decide to give it a try, I hope you can find something in it, too!