I had a 100-400 and a 70-200f4. Both nice lenses. I wasn't keen on the 100-400 as I really didn't like the push pull zoom, which does act a little like a vacuum cleaner as dust does get inside eventually. I aslo found "my one" a little soft at the long end wide open. Others seem to be better so maybe mine was a bad apple.
The 70-200f4 is a great lens, light and produces great results, but limited on it's own.
I sold both and bought a 70-200f2.8 IS
and a 300 prime. The 70-200f2.8 is an excellent lens that I use a lot with and without the 1.4x extender. I did have a Sigma 300 f2.8 that produced great results but mine fell apart after 5 months and I managed to take it back recently and now have a Canon 300 f4 IS
In your situation I'd keep the 70-200 and if you want to get the longer reach buy the 300f4 IS
. It's much lighter than the 100-400, and produces better results at 300 and 420 when a 1.4x extender is added.