The level shifter sounds neat. I take it the LV side would be the 3.3v and the HV the 5v? 3.3v pwm from the PCA to A1-A4 and 5v pwm coming out on B1-B4? 5v to the HV pin? 3.3v to the LV?I was looking at an image of the board but doesn't look like they gave the resistor array an identifier so a little hard to know for sure without a continuity test. It's one part and really not that hard to remove, just alternate heating both sides quickly and it'll eventually slide off.
You could use something like this if you want 5v and not remove the resistor.
4-channel I2C-safe Bi-directional Logic Level Converter
Because the Arduino (and Basic Stamp) are 5V devices, and most modern sensors, displays, flashcards, and modes are 3.3V-only, many makers find that they need to perform level ...www.adafruit.com
I don't think it would damage the Pi immediately as it isn't too high of voltage but long term it could.
EDIT: Pretty sure it's the resistors I circled as it looks like two go to ground and 2 to VCC.