Riding an ATV on a Mississippi highway, on public roads, or within city limits is illegal. Also, you should avoid riding on paved roads, as ATVs aren't built for pavement and can easily turn over during turns.
Check out these Caledonia, MS ATV/OHV resources: