Mould composition DOES matter, quite a bit
I've now cast in aluminum, steel, and brass moulds. The mould's composition matters. The 650 deg. F. temperature works well for steel ones. However, on average, my Lee aluminum 6-cavity moulds prefer 700-750, and I have one whose sweet spot is 800 deg. Fortunately, with the aluminum moulds, you can use the trick of a wet sponge to periodically cool down the mould and sprue plate, and I can cast boatloads of boolits in short order with them.
Try out different temperatures, and if you need to, write down each mould's favoured temperature on a sheet of paper. It shouldn't take you long to figure out the sweet spots.