This is easy to do. A CMOS 40192 counter should work. It has separate UP/DOWN clock inputs which would be the "conditioned" inputs from your switches.
It outputs BCD (binary coded decimal), or 4 bits used to indicate a number from 1-10. To get a single output you can connect these 4 BCD outputs to the BCD inputs of a CMOS 4028 decoder, which is a 1 of 10 decoder and has 10 individual outputs, only one of which would be on at any time.
Normally the 40192 would keep counting up/down when your switches are pressed (when it hits 10 it goes back to 1 and vice/versa). To make sure it stops at 1 or 4 (0 or 3, actually, since the counter goes from 0-9, not 1-10) you need a couple of invertors and AND gates to inhibit the clock inputs from your up/down swiches when you've reached either "end" of your count.
If you pick up a beginners book on CMOS and/or digital electronics you'll find it's pretty simple stuff (when dealing with "hobby" type circuits). What I've typed might not make much sense, but it really is quite simple once you understand logic and gates.
This circuit could be done with only 4 IC's.