I bought an 80 lb bag of quick-concrete mix at Home Depot for about $25. I put the un-opened package into a trash bag, then into a burlap sack.
Since its in the heavy duty burlap sack, its easy to take in and out the the sidecar when you want to.
One of my winter projects is going to be to put a deep cycle RV battery into my sidecar trunk, wired into a marine battery selector switch. I want the extra battery for powering stuff while on camping trips, but a size 31 deep cycle lead acid battery weighs about 60 lbs... so i guess it counts as ballast too. I'm using a marine battery tray with quick release straps and my battery selector has quick release clips for the wires too, so I'll be able to take the battery out in just a minute when I want to free up trunk space.