--Please improve directions to swimming holes if you can--
Simon's Rock is pretty much surrounded by beautiful, natural places to go swimming. At a lot of them you can get away with swimming naked. Some you have to trespass to get into.
Green River: Most treasured among the swimming holes is Green River. In days of yore, Barbara Resnik was known to take her workshop group here, and folks jumped off the bridge in their underwear (assuming, and at Simon's Rock this is a big assumption, that they wore any...). Green River is close enough that you can walk to it between classes (or instead of a class). Or, it used to be before Elizabeth Blodgett-Hall's daughter took the footbridge behind the ARC out and forbade Simon's Rocker's to go there. Some rockers have a different favorite way to get there and will go whenever the opportunity presesnts itself. Here's how to get there without her calling the cops on you: Go at night (it's gorgeous). Take the long walk around back of the Labarynth by road (about 20 minutes). Cross through the Labyrinth, then you can pretty much follow the line of the corn field up until you get to the path to the river. Make sure the river's deep enough before you jump off the bridge, and put your clothes down on the rocks so she can't throw them off into the water if you make too much noise and she notices you. Simon's Rock students have jumped off that bridge every month of the year (yes, even December). It's a spiritual experience, and there's no reason to miss it if you're a bit clever. Last tip: be respectful; don't leave crap lying around out there and do try not to raise a ruckus. It's too beautiful to waste. Here's a map to the walk you need to take to get into the Labyrinth from behind. During warmer months, the Green River often experiences problems with high e. coli counts. When this happens, try the Williams River or the Black Hole instead.
The Williams River: Close to Simon's Rock, the Williams River swimming spot is located on the property of Rachel Fletcher, a local woman. She allows people to use the spot for swimming and fishing. The river is clean, wide, and shallow, making it a great spot to lay down in the water and relax. One of the best features of this spot, which is located on Division street between Christian Hill Road and Route 41 (look for a pull-off next to a small bridge) is the fact that the entire bed of the stream is made up of soft, clean, light-colored clay which feels pretty awesome and is probably good for art. This spot is also known as the "electric fence spot" because of it's proximity to an electric fence, which I highly suggest touching/daring your friends to touch. It does not experience the same high e-coli counts as the Green River, but is sometimes a little too warm during the summer to really cool you down.
Long Pond: You can get to Long Pond by heading up Christian Hill Road (go past Lake Mansfield on your way to the entrance to upper campus, pass the entrance to campus, make a left onto Christian Hill road) and going straight through the blinking light at the intersection of Christian Hill and Division. Look for a spot just big enough for one car to park and head down the rough trail here to the lake. You can either swim from the shore or go left once you hit the water until you come to a dock which is awesome for swimming off of. The status of Long Pond as private property is unclear. Swimming out to the Island (which almost certainly contains treasure) is fun. This spot can also get a bit too warm in the summer.
Other swimming holes:
The Quarry: A bit of a hike is involved once you get there, but it's well-worth it. Getting here involves going up E. Alford (possible on a bike but car is way less struggle), or you can come up 41. It features really high stuff to jump off of. See google map. 
Townie Hole: You can bike down or else drive and park on the edge of the highway at the bridge. The path leads to several swimsuit spots, then if you wade up river it's more private. Map here. 
Prospect Lake: This is a 15 minute pedal or a 5 minute drive. Here it is on a map.
Bash Bish: It's a park. You can swim there but it'll be cold. People do go skinny dipping here.
"The Black Hole," in Monterey: A deep, turbulent pool of murky but clean water in Monterey. It is fed by a fast-flowing stream and is one of the first swimming holes to be pleasantly warm in the spring. It's private enough to be clothing-optional, but as it is frequented by families, you ought to bring some clothes just in case. It features a good rope swing that you can get a decent flip off of and a clay bank on the far side, which people who are in the process of swinging in don't appreciate having clumps of thrown at. There's almost certainly treasure at the bottom, and if you stay in the same place without moving for too long, catfish will nibble on you.
Lake Mansfield: We all know where it is, and it's usually awful; too many reeds, too parklike.