Surely the vast majority of the traffic via ZRH is longhaul anyway. Most destinations are covered by GVA already with direct flights so there would be no need to transit elsewhere for the regular economy passenger.
Worth pointing out though that although GVA is close to the city, you either need to pay for parking (at exorbitant cost) or hire a taxi regardless of where you're coming from. It's surely easier to catch a train from the city centre if you do choose to transit.
You're right in so far, that we are talking here mostly about destinations that have no nonstop flight to Geneva and there mostly long haul, so the leisure pax that do not like to transit are out of this equation anyway. But there is a big mistake in the picture: you can catch a train from the city center - and be at GVA airport in six [6!] minutes, without any parking fees at all. At the moment one has the choice to transit in ZRH, FRA, CDG, LHR and also places like DXB and a couple more I forgot about. If Zurich is out of the equation, there are still plenty of options.
A 3 hour train ride is a perfectly fine replacement for a P2P traveller, but when it comes to connections the train starts to loose after little more than an hour. One need to be at the airport like 90 minutes in advance anyway - but would one really consider three hours in a train as more convenient and more comfortable, than a 45 minute flight and a 45 minute transit? To believe, trains are a good replacement on that route for people heading elsewhere than Zurich, is the digested remainders of cattle farming, to stay polite. It is necessary to know, that Switzerlands "long haul" trains cannot be compared to, let's say an Eurostar or TGV. They are little more than a faster-than-usual S-Bahn in terms of comfort and service (I use them ever day commuting to work, so let's consider this opinion as experience-based
To put some numbers in the game: last year there were 315'581 passengers from GVA to ZRH and 313'039 in the opposite direction. From those were a mere 40'552 and 15'765 actual domestic travellers per direction. And how many of them might have been non-revs, read deadheading Swiss crews? As one can see, the domestic passengers already use the train. If just even 20% of todays transit pax actually will take the train in future (which I doubt that it will be so much), there are still short of a half million passengers no longer flying LX, but rather LH in the best case. But also the likes of AF-KLM, British Airways and of course the ME3 will be pleased to take their share.