If your writing makes you smile, write it.
If your writing makes you think, write it.
If your writing keeps you up at night because you can't get it out of your head, write it.
If your writing turns you on, write it.
If your writing makes you sweat, write it.
If your writing makes your heart race, write it.
You don't need anyone's permission to enjoy writing something. You don't need to cater to an audience you don't already have. Or even one you do have. You don't need to keep to a specific niche because you know your readers like it.
Is that new genre calling out to you but you've never written it, so you're uncertain how it'll turn out? Write it.
Challenge yourself. Have fun. Create create create.
The only one that matters in the end is you. Audiences come and they go. People may be upset if you change what you do, but in the end, do you like it more? Then keep at it.
What you do should make you happy. You'll find a new audience. There'll be an audience for your new writing. Just keep pushing forward.