This is for Stich

because I didn't want to post a really long comment, and more interested people are likely to see it here.
Also I'm going to apologize in advance for the number of parentheses you're about to encounter.
I should preface any advice with... I'm new as hell to the "knowing anything about computers" club and most of what I'm about to tell you is second hand knowledge from my own personal research, not experience. You can probably put together a pc capable of running any game out today (though most probably not at the higher graphics settings) starting in the $600-$700 range. If you're interested this is a list of the components that went into my computer (plus a DVD burner which was about $20 and legit copy of windows 7 for $200 (ouch)).
There are 8 basic components that you need to have for a functional computer:
1)CPU aka Processor
2)GPU aka Graphics card
3)RAM
4)Motherboard
5)Hard Drive (or a solid state drive (or both) but those are kinda (really) spendy still)
6)Optical Drive
7)Case
8)Power Supply

about:
1)The CPU is the brain of the computer. The two main brands that make processors are AMD and Intel. From what I've gathered, both put out a quality product that will work well and last. Intel's 2nd generation Core (i3, i5, i7, if you see the term Sandy Bridge, it refers to the dual channel memory technology in this generation of processors) are generally seen as performing better and faster than AMD's comparable chips, but you pay a bit of a premium for the Intel name. Just about any quad-core processor will be able to handle today's games and will probably be just fine for quite some time. I would recommend one of the Intel i5s or AMD Phenom II x4s numbered 955 or higher.

2)This item is actually non-essential. If you don't do any photo/video editing or watch ultra high definition videos or play any serious games, the graphics capabilities of the CPU and motherboard will be enough. However, this component will be the determining factor of how good your games will look while providing an acceptable framerate. There are two main companies that make GPU chips (which go into graphics cards that are manufactured by a variety of companies), AMD (under the brand Radeon) and NVIDIA. Both have very good reputations with their slightly different strengths. If you see the term SLI/Crossfire, that just means linking two of the same cards together to increase performance. This is a link to an article that shows how all of the current graphics cards perform in certain tests and games. Guru3d.com and Tom's Hardware have probably reviewed every single GPU on the market. Do some research. Decide what kind of money you have to spend and what kind of performance you want.

3)Get 4 GB of it. or more if you want. I was recommended the G.Skill Ripjaw series. I know Corsair and Crucial are very reputable brands. I'm sure others make good stuff too. In the specs you should see DDR3-xxxx. The xxxx is the speed. 1333 is standard and plenty good. Higher numbers are faster. Whatever.

4)If you go with an Intel processor, you'll probably want to go with a motherboard that uses either the p67 or z68 chipset. When you're looking at motherboards, you'll mostly be checking out what connections it has. For example, how many SATA 3.0/6.0 channels it has (drive connections), or whether it has USB 3.0. I know ASUS is very good. I went with Gigabyte. I'm sure MSI and other manufacturers also make good boards.

5)Get a hard drive that runs at 7200 rpm and uses SATA 6 gb/s for the fastest data rates. Choose whatever size you think you will want to store all your files and music and movies and games and such. Western Digital is the king of hard drives. I'm sure others will be just fine though.

6)Get a CD/DVD drive/writer. There are a ton to choose from in the $20 range on newegg.

7)Keep cooling in mind. Proper ventilation is nice. It keeps your stuff at a reasonable temp which helps prolong life and improves performance. Think about getting a case that has features that help with cable management. Don't get the case I got.

8)You might think you can skimp on the power supply as it really isn't a determinant of performance. It just supplies power. But if your power supply sucks or is in-efficient you can run into some problems. Go with a well-respected brand name (I've heard Corsair is pretty much the best), at least 80+ certified if not 80+ bronze or better. If you're not overclocking your CPU and only plan to use one GPU, 600W or 650W should be plenty. I went up to 750 in case I wanted to go the SLI route down the road.

You'll also need a keyboard, mouse and monitor obviously, but you might already have those. You'll also need an operating system (Windows 7). I've heard you can get illegal (free) copies without too much difficulty, but for whatever reason, I decided not to look too hard and just bought the shit.

Feel free to leave questions in the comments.
Sites/videos I used:
pcpartpicker.com - has pretty much everything by everyone. Only lets you pick components that are compatible. Tells you the price on a few different sites (newegg always seems to be the best, although you should shop around. Amazon is actually really good, as is TigerDirect)

cyberpowerpc.com - They will build a computer for you. It will cost a hair more than it would to do it yourself, but you may want to look into that option. I used the site to get an idea of what I would want to use, about how much it would cost, and how much power I would need

guru3d.com - tons of info and tests and charts. Where I did most of my GPU research.

Tom's Hardware - great site. tons of reviews and how-tos. very helpful forum community. I ran into an issue during my build and posted a question in the forums. the people there helped me solve it in half an hour on a friday afternoon.

tested.com - more reviews and how-tos. They have a couple of pc build videos if you're interested.

This guy is talking about building a gaming pc.

A series of videos walking through building a computer.