What would you use to collect these rays? In this link read the part called, "Matter Interaction", about half way down the page. https://en.wikipedia.org/wiki/Gamma_ray
The ideal photon energy is enough to bump an electron to a different energy level, but still within the atom.
When you get to "ionising radiation" energy levels, electrons are knocked clear out of the atoms, which does not give any photoelectric result.
The implication is that the silicon would start conduct, I think? Producing free electrons and "holes" should be a bit like injecting charge carriers as done by the base current of a bipolar transistor.
edit - Thinking about it more, EPROMs can be erased by exposure to X-Rays. That's presumably the same effect as I was speculating about above, the ionising radiation making the silicon "leaky" and discharging the EPROM buried caps.
Gamma rays contain more energy, but this isn't the same thing as turning the energy into electricity since you also need the correct materials. Aside from being high energy enough to eventually mess up the finely tuned properties of whatever material you were using to produce electricity from them, gamma rays also pass through most materials easily so. So if whatever material you were using was not dense enough (like lead or something else) to catch most of the gamma rays, you would need a lot of it so that fewer gamma rays could pass through it without being absorbed by the panel.
I don't think exposing a regular solar panel (designed for visible light) to gamma rays will do very much. A lot of gamma rays would pass through it and those that don't would cause cumulative damage the solar panel. You would need a panel that specifically targets that part of the spectrum.
EDIT: I got curious and started searching around. Apparently making a thicker cell to absorb more gamma rays that easily pass through most materials won't work. I ran into this thread: https://www.physicsforums.com/threads/can-photovoltaic-cells-capture-gamma-photons.538569/
It states that the reason you don't have gamma ray photocells is because they have to be too thick since gamma rays pass through matter so easily. But the thickness required is thicker than the "electron escape depth" so any electrons that are knocked loose lose all their energy before they can migrate to the surface of the material. This is required because photovoltaic cells work by having a light side (front) and a dark side (back). The light side causes electrons to be knocked loose and electrons on the dark side migrate towards the light side which produces the charge imbalance that is harnessed for energy.
If there is another mechanism or material, humans don't seem to have discovered it yet. Except of course, by just using the heat generated.
I guess that counts. Those function off the heat generated from impact though and carry their own onboard source of gamma rays. I was under the impression that the OP was wondering about using gamma rays from external sources. Same method would technically would work...but it'd have get very very hot to be effective and if you were close enough to a star to do that then you're also close enough to use solar panels (I think at least) which are more effective and efficient.
There are crazy temperature differential in space between light and dark sides though and we still use photovoltaic effect rather than the Seebeck effect. Is the Seebeck effect really that much less efficient than the photovoltaic effect even when those temperature gradients are present? Maybe solar panels are just cheaper? But it's hard to believe that growing and processing silicon crystals is less expensive than connecting two thin sheets of different metals together. It seem like it would certainly be more rugged than solar panels.
gamma ray photons are more likely to damage a typical solar panel. NASA has done a lot of the theoretical and practical work already. in orbit, solar panels are subjected to a lot of energetic particles, gamma rays, X-rays, and cosmic rays. semiconductors in that environment can undergo permanent changes in their electronic characteristics (including shorts). semiconductors used in space are radiation hardened to prevent such failures (which is why NASA buys hardened 2N2222 transistors for $50.00 each).