How about the bearing shells, could they be also mated to the case/crank? BMW does stuff like that.
Someone somewhere found out how time consuming and expensive it can be sorting that out of 2-3 engines and boxes of parts.
So...if we ignore the case damages for now would just pulling that pin out and installing a new one suffice to keep the oil where it belongs?
How big is that hole and is there a lip on the bottom, as in the pin not pushed down far enough to contact the case? Then maybe rig some kind of hammer puller, a small one and carefully bang it out.
Depending on how tight that pin is, can't be very tight if it walked in there? How did that happen anyway? If you pushed it in by mistake, then you'd know how tight it is.
Maybe just roughening up the inside with the proper dremel bit and then installing a bolt/anchor in that hole could be enough to provide some pull.The proper bit probably wouldn't be in the accessories that usually come with them dremels.
Now that you have that engine all apart, did you ever find out why it was knocking? Starting to look expensive, is that high milleage bike worth all that expense of time and money?