Data Mining on the World Wide Web

Are you writing a paper on solar power and you want to find out how many companies produced solar thermal collectors from 1980 to 1990? Does your boss want to know how many barrels of oil were produced in Richardson County in 1978? Perhaps you want to compare Nebraska's energy consumption to that of other states? Answers to these and many other questions are closer than you might think.

There are countless web sites that offer energy-related data. Government web sites have a lot of information and data that can assist in your research. Many of these sites also contain reports on conservation, production and consumption trends as well as numerous other subjects.

For the most recent state and local energy data, access the Nebraska Energy Office's web site at Currently, the web site is getting a major face lift. The agency's web site displays data and information in a variety of areas such as energy prices and consumption by type of energy within economic sectors.

There is also a page with "links" to the Department of Energy's Energy Information Administration, or access it directly at, where you will find a wealth of data and useful reports. This site offers the most current national energy statistics available, except for state-specific data which can lag three to four years behind national figures.

Honing the Search

The large number of web sites and the varying quality of information can make using the web for research a daunting proposition. How can you find other web pages that pertain to the subject you are researching?

There are several "search engines" available that allow you to find web sites that contain information for which you are looking. To use a search engine you simply type in the words (e.g., a name) of what you are trying to find. These search engines do a good job of finding web pages that contain these words. Unfortunately, they do not necessarily distinguish among pages that contain exactly what you want and those that merely have the words you typed. For example, a recent search on a popular search engine for "energy data" provided 200,941 web pages that contain the words "energy" and "data." Clearly, not all of these pages have information relevant to energy data and not all have exactly what you are trying to find. Adding more words to the original search can help reduce the number of web pages the search engine finds. But the way some search engines operate, adding more words will increase the number of sites found!

What can you do? The key to using a search engine to find specific information is to be a smart search engine user. Most search engines allow you to customize how your study is conducted. Furthermore, the better search engines provide users tips and other tools that can assist in targeting information. Look for a "search options" button or link that allows you to enhance your quest. There are also "composite" search engines that scan several general search engines simultaneously! This can be a useful first step to help you find specific information and eliminate useless web sites.


Search Engine Tips

Use ideas and concepts instead of just keywords.
Use more than one word in your search.
Use descriptive, specific words as opposed to general ones.
Explore ways to employ advanced search options.
Know when to say when. Limit the amount of time you spend "surfing" for information.
Know when to ask for help.
Keep in mind that some information may not be available on the web.


General Search Engines

Excite at provides plenty of ways to customize and limit your search.

Infoseek at is a solid search engine that is very user friendly.

Lycos at has many advanced customization options through its Lycos "pro" option.

Magellan at is a very good search engine that can be extensively customized.

Yahoo at is the industry standard and fun to say.

Webcrawler at is one of the first general search engines.


Composite Search Engines

Highway 61 at is easy to use.

Metacrawler at is fast and effective.

Motherload at has a strange name, but is a great search engine.

Profusion at allows you choose specific search engines


The Energy Office's and Energy Information Administration's web sites are listed on this page, as well as some tips and a list of popular search engines. If you have questions about gathering energy data and finding other energy information on the web, please contact Jonathan Strand in the Energy Office, phone 402-471-3538, or email Happy hunting!


returnfall98.gif (3789 bytes)