I think you’re on the right track. Rose mentions that her home world was stripped of its natural resources, and she implies those resources were either used to rebuild their navy or used to pay for it. There does seem to be an implication that the First Order was able to strip planets of their resources throughout the Unknown Regions unopposed once they learned how to navigate it (with the help of Snoke apparently).

Also, as Dom mentioned, there were those in the Republic that also supported a First Order-coup. Either they believed in the cause or financially benefited from the Old Imperial status quo, so they also were possibly helping fund the First Order.

I believe the novel Bloodline also mentions that the First Order was being funneled money from crime syndicates through various shadow corporations as well, possibly because they believe the First Order would allow them to operate with more freedom than the Republic does, much like the Empire did before it.