What type of starter motor do you have? Probably a Velo - these are smaller and more efficient than the older bosh ones. They also have glued in magnets - the glue fails and the magnets grip the rotor causing all sorts of problems ... when you put the car battery on you simply increases the amount of force applied ... possibly doing more damage.
What you can do is put the bike in top gear, bike on centre stand and turn the engine over using the rear wheel - this will confirm that the motor is not sized. If you can get the battery checked - it is probably ok. If it is the starter motor - you will need to disassemble it - do a search.