I'm assuming that you are referring to the reservoirs in Oxford.
As long as you are in compliance with the applicable boating regulations (i.e. wearing a PFD during the period of October 1 to May 31) you are good to go. If there was a posted swim area there (there is not) you could not go into the swim area with a kayak.

If you are going to carry your kayak in, I would suggest the northernmost lake by the power lines. It's a short carry. The other three lakes are a pretty good distance from the parking area.