Any sufficiently advanced technology is indistinguishable from magic.
- Arthur C Clarke

There's a reason that science fiction and fantasy are frequently shelved together - separating the two is usually a fools errand.

The Dragonriders of Pern features a preindustrial society where flying, firebreathing, teleporting, and telepathic dragons defend the skies from horrible creatures that rain down from above. They could easily be considered pure fantasy - up until the book where they discover the spacecraft that the human ancestors flew in on, and the labs where they bioengineered the dragons.

In Star Wars (arguably the most famous science fiction property of all time), the Force is really just space magic called another name. And they do all their fighting with swords.

The lists go on and on. Science fiction and fantasy are united in that they explore the impossible. Fantasy uses elements that will never be possible, while science fiction uses elements that theoretically might be possible. But the impossibility is shared.

Science fiction and fantasy are not a single genre - they are many

Epic fantasy (eg Wheel of Time) and space opera (eg Star Wars) are more similar to each other than they are to urban fantasy or "hard" science fiction.

In some ways, it's inaccurate to call science fiction and fantasy genres at all. They're setting elements. A romance doesn't stop being a romance because it's set on a space station, and a murder mystery isn't any less mysterious because it was an elf who was murdered.

The key to mixing these elements successfully is to understand the expectations of your readers, and meeting those expectations (but in surprising ways). Genre helps define expectations.

If you're writing hard SF, readers expect to see a world that's close enough to our own that they can believe that our world could become the world of the story - magic obviously has no place here.

On the flip side, urban fantasy readers expect the author to have considered the modern world and how it might interact with the impossible. Extrapolations from science to science fiction may be natural depending on the exact nature of the setting.

If the reader has led to expect that the serial killer is a Scooby-Doo villain, then the sudden reveal that the murders were done with actual magic will violate their expectations and frustrate them (in general - there will of course be exceptions). But if the story is framed as a space cop trying to capture a vampire despite not believing that vampires are real, the readers will accept that premise as well as any other.

The reason this is often recommended against is because by mixing them you find yourself unable to meet certain genre conventions. Fantasy readers want swords and lords, sci-fi readers want spaceships and aliens. Even more importantly fantasy readers want epic tales of good versus evil, while sci-fi often want to grapple with trans humanist ideas. But trying to match both markets, you match neither.

That said, I don't think it is very good advice. It is not the writer's job to market, it is their job to write a good story. If you have a good story that involves fantasy and sci-fi elements, write it.

Plenty of existing works mix genres to varying degrees and been successful. Star Wars is mostly sci-fi with a little fantasy, while the Shannara books are fantasy with some subtle sci-fi.

Mixing genres can be a challenge, but don't limit yourself by blindly following a rule.

You are correct that you are not the first to attempt this blending. The entire Urban Fantasy genre does an admirable job of blending today's scientific reality (which was yesterday's science fiction) with classical fantasy elements. Many science fiction subgenres such as Gothic SciFi, Space Horror and Slipstream exist specifically to categorize different approaches to such blending. Best selling authors such as Dean Koontz, Steven King, Raymond Feist, Mercedes Lackey, Clive Barker, Neil Gaiman, Kim Harrison, Laurell Hamilton, Peter Straub, S.M. Sterling and George Martin (along with many many others) have at least one book which bridges the gap between fantasy and science fiction.

Whoever cautioned you against blending is ignoring this major trend in the modern fiction market. With all due respect, I couldn't disagree with them more.

There are challenges to successfully merging fantasy and science fiction. As @RTPax noted, there are genre conventions which must be respected as you mix your story together. But those conventions are not mutually exclusive. There are many scientifically plausible explanations for magic existing in an apparently physical-law-limited world. Mercedes Lackey suggests in her Razor's Edge series that the Elves have been hiding until just recently, when the rise of aluminum and alloy metals reduced the amount of (fatal to elves) raw iron exposed in our day to day lives. S.M. Stirling blames us humans for breaking our previously functional universe though our high energy physics experimentation, opening the door for all types of unexplainable events. You will need to find a reason for why magic now exists in your scientific world, or why the world still believes in science in the presence of magic which defies cause and effect.

Writing a merge is not easy. It requires that you wear two hats while you are writing so that the requirements of each genre can be filled. You must also be careful during editing, not to leave that hard-earned fulfillment on the cutting room floor.

But if it is what you are called to write, and if you are up for the challenge, the results can be spectacular.

Keep Writing!

A lot of wonderful books combine both science-fiction and fantasy.

And why not?

If magic can exist in books set in the modern age or in the past, why not in books set in an otherwise realistic future?

Why can't mythical creatures go to space?

What reason is there to ban ghosts and purveyors of the supernatural from a world of great technology?

One of the most common crossovers involves time travel. Time travel can be either science-fiction or fantasy, depending on how it's used and what other elements are present. If you're Wesley Chu (Time Salvager), it's 100% science-fiction. If you're Deborah Harkness (A Discovery of Witches), it's absolutely fantasy.

Why do people advise against blending the two?

My guess is because most readers who enjoy one genre eschew the other. Or at least that's the belief. I love both and so do a lot of readers. But if you write both together, you run the risk of turning off a subset of potential readers. Of course, you also have a lot to gain from crossover readers.

Write what you love. With luck, the audience will follow.

Examples: Dune (series), Ender's Game (series), The Bone Season (series), The Golden Compass (series), A Wrinkle in Time, Star Wars (movie series).

If not done well, blending genres can ruin the suspension of disbelief required for enjoying either science fiction or fantasy. Typically, for a speculative fiction book, the reader absorbs some baseline rules for this setting, internalizes them, and then stops thinking about them, in favor of paying attention to the story. Usually you want this to happen pretty quickly, and in the case of a genre with set expectations, a lot of this work has already been done for you by previous writers.

Finding out that the rules of the world are different than what you thought they were can be a legitimate and compelling plotline. But unless that is your focus, you don't want to call attention to it. When one suspension of disbelief piles on top of the other, it can lead to disbelief fatigue. For instance, in the movie Looper, the main concept is time loops, traveled by assassins. For me at least, the addition of psychic powers to the mix threatened my sense of immersion in the story. The two plot elements didn't have any natural connection, so they had to be swallowed separately.

That isn't to say blending genres can't be done wonderfully and well (many of the other answers have given great examples). But your question was about the reasons to NOT do it.

I'm of the opinion that "Don't Mix" should be seen more as a caution than a prohibition. A very useful caution, but a caution nonetheless.

Hard sci-fi and pure fantasy work with very different world-views - one embraces technology and the other magic. Technology works within the framework of physical law, where what you want is entirely beside the point. Magic is (traditionally) based on a rejection of physical law, and the desires/will of the magician are what count. Technology is above all impersonal: predictable, although often obscure - Mother Nature is a coy lady whose secrets need effort and intelligence to uncover. Magic is (or was) fundamentally personal, full of irrationality and wonder. Attempting to mix the two runs into the strong temptation to mix the worst of both worlds, producing either technology which is indistinguishable from wish-fulfillment, or magic which is, well, boring.

Over the last several decades, there has been a tendency for fantasy to be infected by technological influences. I personally blame Larry Niven's "The Magic Goes Away" for starting the movement, with his concept of magic being limited by a resource (mana) which can be used up and never replenished, very much like, say, oil reserves. While modern fantasy can produce good stuff, many writers seem to treat magic as just another skill, just another set of rules to be obeyed. There is an impersonal quality to the practice, and magic is not dark, irrational and dangerous. Magicians tend to be more like mechanics than wizards.

In other words, the magic gets sucked out of magic.

While the converse is also to some degree true of technology, the danger has always been there, since the author can simply describe almost any level of effect and invoke "advanced" concepts to justify it. As Arcanist Lupus pointed out, Arthur C. Clarke identified the problem at least 45 years ago, although to be fair, he was referring to real technology, rather than fictional, so the meaning is rather different.

One way to look at the problem is to consider dramatic tension, which is the heart of most stories. Stories are about protagonists overcoming obstacles. If a knight battling a dragon can whip out a heavy machine gun and shoot it out of the sky, it takes a lot to make the story interesting. If a scientist trying to invent a serum to stop a plague can call up a demon to do the job, well, who cares?

The trick in either case is one of tone, plot, and ingenuity. Crossovers can work. It's just that striking the balance gets harder, since crossovers need to address the rules of both schools simultaneously - or break them simultaneously, if you prefer. Either way, it's tricky to do well.

I would just make these a comment since I am only recommending two that I have not seen yet, but I don't have enough rep to comment on the question.

I would check out Warhammer40k and it's extensive universe with many stories for inspiration on how to blend fantasy and sci fi. They have magic users that access something called the Warp while they also have mech troopers and space ships. I am not vary familiar with it but it is popular

The second source you could check out it Starfinder with any published adventure. While it is a table top game, the published adventures tell a story, and the universe has a good mix of magic and science fiction. I just started reading the Core book so I don't have good examples!

From these two examples, you can see they are not hard science fiction combined with fantasy. I think most people recommend separating those two, not the general fantasy and science fiction genres. People may argue that magical advances will halt scientific advances since magic will handle problems in the universe. I think that is the center of the argument against "combining" the two. However, as stated by many of the answers above, fantasy and science fiction are very diverse on their own and the blending of fantasy and science fiction is actually done numerous times successfully.

"A [space] wizard goes on a quest to save a princess from an evil empire."

Take out space, and I would think we are talking about some Forgotten Realms story. So, moral of the answer is, there are equal audiences on both sides of combining or seperating sci fi and fantasy.

Both Magic and Science provide constraints.

I think one problem with mixing magic and science fiction is that science fiction readers prefer some whiff of plausibility in however science is extended for the purpose of the story; so you run the risk of disappointing those readers by just using "magic" when you don't know the science.

And vice-versa! Magic, like science, is expected to be a tool, with constraints on how it is used. Your story cannot just be filled with deus-ex-machinas because they are convenient for you as an author. You can't just hand-wave everything away as "magic", or your heroes aren't heroic at all.

Remember you are writing a story. In order to be a good story, the hero must solve a problem and struggle to do that; they can't just wave their wand and solve the problem on page 1. Or page 100! Readers keep reading to find out what happens, which means the outcome (in the next few pages, by the end of this chapter, by the end of the section [Act], by the end of the book) has to feel up in the air. That means you need constraints on how the hero can solve the problems, and these need to feel (to the reader) near crippling; so they are kept wondering how in the world the hero will get it done.

Your magic system cannot be "anything goes," that just kills the suspense. Your scifi cannot be "anything is possible," either. For the same reason. No suspense.

By combining them, you run the risk of loosening the constraints (just use magic if you can't use scifi), and thus killing the suspense.

That is why fantasy stories have limitations on what magic can do, and why scifi stories have limitations on what science can do. In both cases, they allow non-realist stories to take place, but are actually obstacles that must still be overcome by human ingenuity and spirit. Harry Potter and his crew must still be brave and risk their lives in the face of lethal danger, because magic alone isn't enough.

There's no problem with mixing. In fact, it's the norm. That's because sci-fi tech and magic are both shorthand tropes to accomplish many of the same things.

Consider Star Trek. Even TNG, which is the "hardest" of the canon, has various forms of biological telepathy. If pressed, their official explanation will use scientific language. But that's just the flavor. For example, "Betazoids were natural telepaths, an ability centered in their paracortex, with psilosynine being a main neurotransmitter" 1 is no different than "Deep elves were blessed by their volcano god to have telepathy." In terms of accommodating the supernatural into your story, it's the same effect either way.

Similarly, almost all fantasy uses technology which is not only beyond the established mechanical engineering of the world, but is often beyond modern engineering methods (or flat out impossible, though not considered magical). Again, it'll be described in narrative appropriate ways, such as "gnome/dwarf/elf", "ancient/forgotten/forbidden", use impossible materials like adamantium, Orichalcum, etc. Admittedly, that rarely involves spaceships and lasers. But it regularly involves otherwise impossible travel and weapons. Again, different words for the same effect and necessary considerations for the story.

So you see there's a natural attraction between sci-fi and fantasy. The challenge becomes not blind-siding your reader. For example, if the final chapter of your heretofore "pure" fantasy book ends with the dwarves launching their mountain into space to colonize a new world, that's almost certainly a problem. As with any significant event in your book, it should be at once a surprise but also expected (or, at least, expectable).