you'd waste a great amount of energy.
you could use a simple booster circuit, based on single transistor.
then you put all the LED in series!
the booster will act as current source (in my simple circuit there is no regulation at all).
small 2n3906 can drive 5 or 6 low-current LEDs from 2.4 volts.
if you need more current use a different transistor.
i mean, something like this (in the attachment). they are all in series!
parallel connection can result in brightness variation.
but you do not write the type of LED.
the booster also will waste energy. but you can use 5 LEDs or 7 LEDs, and it will work, without modification. while if you use 12 volts, you still get a limit of 5 or 6 LEDs.
i found some circuits on talkingelectronics.com
and then tried modified variations of them for some months.
but I am not really a complete expert.
I just needed to step up 5 volts to 6 volts, without to have complicate ICs or regulation schemes.
maybe such a booster would be a solution for your task.
you can not connect LEDs of different color in parallel, as you maybe already know.