I dont' know specifically about your uC, but on the PIC micro, the weak pull-ups are somewheres around 200kohm+, certainly not enough for I2c, provided they even work on your IO pins when the i2c module is active ... so placing 4.7k in parallel with that, the 4.7k will look like a short compared to the 200k ... in this example, you get about 4.5k, which should be fine.
with i2c the idea is to have a strong pull-up to Vdd, not a weak one ... 4.7k is the spec, but I've seen even 1k recommended in some cases.
anyway, with the I2C, can we see the code you're using? ... i2c is almost always a write before read ... you need to issue a write command with the read bit clear, and then another command with the read bit read set, and then read the bus to get the slaves reply
the datasheet for your gizmo claims the device behaves like a 24 series serial eeprom, so Nigels tutorials on eeproms would probably have some useful info, even with your uC being different.