[ nitpicking ] ought to be a switch these days [ /nitpicking ]
But yes, it is very much possible. LAN switches are cheap and I'd have yet to see a DSL router that doesn't allow a switch to be connected to it. What you're asking about, Tom12, is exactly the kind of setup I installed in my parents' home years ago.
One word of advice though: be mindful of your wiring. What I used was regular CAT5 and due to the length of the cable from the ground floor to the first floor, there is a lot of noise in the signal. The speed is very much fine for sharing the internet connection, but copying files can certainly take a while longer than you'd expect from a 100Base-TX LAN.
Walk together, talk together all ye peoples of the earth. Then, and only then, shall ye have peace.