What kind of engine should i put in go-kart?

I want a off road veicle that can go 50-60 MPH or higher. I'm going to get a honda engine but how big should it be. It has to be able to get through mud to.

Speeds like that you start with the safety stuff. Roll cage 4 point harnesses etc. Then traction. 4wd with centre diff lock. You don't need a lot if power. Torque is more important.
You could just get a land rover defender or discovery 200tdi or 300tdi.
That would just need a roll cage.

A 50+mph buggy, the last thing you should worry about is the engine. Here's the basics.

Mud you want flotation. That means wide tires and a light frame. Paddle tires would be best for a 2WD buggy.
You will need GOOD shocks to handle offroad at those speeds. Fox Racing is a good start, but you could spend upwards of $1000 each on them.

To get 60mph, you will need a transmission, you can't just run a direct drive. You'll have to look into getting something from a dirtbike or ATV, and a 125+cc dirtbike engine would be the absolute minimum (though I doubt that size would get you to 60).

If you want an off road vehicle that can do 60mph then you need to be thinking of a buggy or quad. A go kart isn't going to be remotely viable.

Engine wise you'll need to look at 250cc motorcycle engines or bigger, or 1litre car engines. A simple chain drive is all that'd be required. Proper kart engines are around 125cc and put out around 30bhp but these would be unsuitable for your purpose.

Biggest issue off road is suspension. If you don't have any, or decent, suspension, any bumps over 20mph will straight out bend and break the buggy. Getting through mud will require grip not power. You'll need to sort out some decent off road buggy/quad tyres.