Alright, sorry it's taken me a while to get back to you. First off, I am by no means an expert on kayaking, but I researched for almost two years before I finally broke down and bought one. I have a Wilderness Systems Tarpon 100 and I bought Kate a Perception Tribe, both of which are 10 foot long, and both are sit on tops. From the get go, I decided I wanted to buy a sit on top, because they are impossible to sink. My balance isn't the greatest, and I figured I would be going over the side at some point. The first day I took it out, I decided to test it's stability, and ended up flipping it. I was able to climb back into it in water where I couldn't touch the bottom. There is no way you could do that with a sit in. With a SOT, there is no cockpit to sit in, as you literally sit on the top of the kayak. There are port holes throughout the kayak that allow any water taken on to drain back through to the lake or pond or whatever you are fishing on. These ports can also be plugged, if you don't want water coming back up through from the bottom. With a SOT, it tends to be a little easier to mount equipment to the boat, and also allows for storage that is more easily accessible. The SOT's tend to weight a little more than a sit in, and you tend to get wetter with a SOT, so if you are planning on fishing a lot of cold weather, this is something to consider. Also, with some of the SOT's, they are wide and stable enough to stand in. I've tried it with mine, and was able to stand, but I will guarantee if I would have set the hook on a fish, I would have been over the side.
As far as price is concerned, you will get what you pay for. Seating with a good lumbar support is a must. There are aftermarket seats out there that are great, but many companies are incorporating great seats with solid back support stock on their kayaks. Something else to look for is gear mounting options. My kayak came with gear tracks that allow me to mount just about anything to my kayak without drilling holes through the plastic. I have mine set up for fishing, and have two rod holders up front, and two tube rod holders in the back, as well as a camera mount up front. Also, size plays into the cost, as well as maneuverability. A longer kayak with tend to track better through the water, making it easier to keep pointed in the direction you want to go. A downfall to a longer kayak though is having a way to haul it.
Really, there's a million different things that you need to consider, but you don't really know what you want until you get one and try it out. If you can find a place that has kayak and canoe rentals, I would suggest seeing if they would let you demo some of what they have. Also, these are great places to pick up used kayaks for a great price.
If you have any specific questions, i would be happy to answer them, but like I said, I am no expert.