There are a bunch of different ways you can approach this, from hosting your own web site on a DSL line to outsourcing the whole thing. And every arrangement in between (for example, you could own the server, but place it in a co-location facility where they handle the network aspects and provide clean power, air conditioning, backup generators, etc).
To start with, you need to decide what you want your site to do. Your question suggests you are new to this and in that case, you definitely do NOT want to own and operate the server. If you're going to sell something on-line, you'll need a catalog, shopping-cart, billing and fulfillment component. If you have no experience with any of those, you're probably best off with a turn-key e-commerce hosting company that can rapidly implement a Web site based on a standard template and provide the necessary support functions.
If your site is just informational, then a simple hosting arrangement could suffice and that could cost as little as $20-25 per month. However, these arrangements will generally not let you do much in the way of CGI scripting or custom programming without considerable additional cost to have someone on the ISP's staff evaluate your scripts for various forms of risk.
As far as space, again, it depends on what you're doing. My personal ISP allows 10meg of hosted Web space, of which I'm currently using about a meg. There is also a limit on how much traffic is permitted to my site (in megabytes per month).But one of the big limitations with this arrangement is that you have to use their naming convention rather than a registered domain name. So for a business, you really don't want to go that way, unless you're just developing a prototype.
The one big piece of advice I would offer is to choose a solid ISP. A bunch have already failed and more will certainly fall.
paul