What math software can I use to write my math book? I will be dealing with plane geometry, graphs, etc. GeoGebra and Scrivener don't give me the experience I want.
All (yes, all) academic mathematicians use LateX to write mathematics for publication. If you're writing a university level math book, you would know that already. Since you don't, I assume you're writing a lower level book but I would be very surprised if your publisher wanted anything other than LaTeX. It's great for you, because you can concentrate on your text and formulae without worrying about fonts, spacing etc. It's great for them, because they can apply whatever style they want to your text.
The one thing you might find lacking in LaTeX is drawing. Technically you can draw pictures in LaTeX, but the tools are not that friendly. So I suggest you focus your search on a good mathematical drawing tool which will allow you to export your pictures in a variety of formats.
Scrivener describes itself as a "word-processing program and outliner". You don't need that, because you have LaTeX.
I concur about the LaTeX remark. Every math content writer I know has learned LaTeX, so its use is broader than academia. There are lots of simpler programs that can take LaTeX equations now. I think it's even in Google Docs. In the long haul, nothing will give the kind of mathematical precision you need like LaTeX will.
I think you might be mostly asking about figures and diagrams. You can probably do just fine with a strong style guide and Photoshop. Make the graphs in any program that has an exportable vector based graphic, then let someone who is good with Photoshop apply your style universally. This way you can get precise images and graphs, but they won't look like they came from any particular source-they'll look like your graphs and images. You'll probably need an illustrator anyway, there are lots of diagrams and pictures in a geometry textbook.