Does your RV normally connect to a 240 volt 50 amp service?
If yes, it sounds to me like whatever adapter you are using from 30 amp 120 volts to your 50 amp plug may not be feeding both legs of your 240 volt RV panel (Lights dim = no inverter power. Refrig not working is on same leg.).
A quick check would be to borrow a known working adapter and plug in with that. If it cures your problem then it is likely the adapter you are using.
A diagram thanks to RoyBraddy.
There is more info posted by him here.
Another possibility is that your inverter and refrigerator share a common 120 volt breaker/circuit or even a common neutral wire which may have problems.