آخرین مطالب
امکانات وب
Chapter 68
Natural language processing
David C. Yen and William S. Davis
The ultimate objective of natural language processing is to allow people to communicate with computers in much the same way they communicate with each other. This chapter briefly introduces key natural language processing concepts and terminology.
68.2 Strengths, weaknesses, and limitations
Natural language processing removes one of the key obstacles that keeps some people from using computers. More specifically, natural language processing facilitates access to a database or a knowledge base, provides a friendly user interface, facilitates language translation and conversion, and increases user productivity by supporting English-like input.
As of mid-1998 when this chapter was written, natural language processing was not yet capable of supporting true conversational input. Most commercially available software limits the number of different users and/or such parameters as the user’s vocabulary, syntax, or speed, and free-form English input must occasionally be supplemented with commands. Most natural language processing software is designed to locate key words first and then interpret the meaning of a sentence or a phrase, which increases programming time and program execution time. Additionally, special equipment is needed to support natural language processing.
Natural language processing is a major area of research within the field of artificial intelligence. It is closely related (either as a front end or a user interface) with expert systems (Chapter 7), and shows great promise as a user interface (Chapter 48). State transition diagrams (Chapter 30) are sometimes used to model natural language processing tasks.
The ultimate objective of natural language processing is to allow people to communicate with computers in much the same way they communicate with each other. This chapter briefly introduces key natural language processing concepts and terminology. A detailed discussion of the underlying technology is beyond the scope of this book.
Natural language processing starts with the input of a string of plain English words (Figure 68.1). The first phase in the process is word recognition. The objective is to restructure the input string as a series of noun phrases, verb phrases, prepositional phrases, adjective phrases, and so on. A state transition diagram (Chapter 30) is sometimes used to model the process.
Next the words and phrases are analyzed to check the integrity of the sentences and to clarify any ambiguities. The knowledge base stores general knowledge (words, linguistic concepts, etc.) and application-specific knowledge. A lexical analyzer is a routine that performs semantic analysis, checking every word in a sentence against the correct spellings stored in the knowledge base and listing all the possible alteative meanings for the sentence. If necessary, an expert system is consulted to deduce the meanings of ambiguous terms and expressions based on context, questions asked earlier in the session, organization-specific rules, and other factors stored in the knowledge base.
|
Once the words are properly defined, a parser routine performs syntactic analysis, essentially diagramming the sentence to form a parse tree. Finally, during the natural language implementation phase, a generator outputs one or more commands based on the meaning deduced from the word meanings and the parse tree. For example, a plain English query might be converted to a set of SQL commands. The computer then executes the commands.
68.4.2 The natural language shell
The natural language processing routine is typically visualized as a shell. The user communicates with the shell by entering plain English character strings. The shell translates the plain English strings into the appropriate commands and passes the commands to an application program. Using a common shell makes more sense than duplicating the same complex logic in multiple application programs.
Speech recognition is an extension of natural language processing. The idea is to use a speech recognition routine (or a chip) to break continuous speech into a string of words, input the string into a natural language processing routine, and then pass the resulting commands to an application program.
One problem with speech recognition is that human language is imprecise and many words have multiple meanings that depend on context. Add multiple languages, dialects, and accents, and the problem becomes very complex. Additionally, few people are skilled at issuing orders or using language with precision.
Natural language processing can support several types of translation. Language-to-language systems translate between two languages; English and Chinese, for example. Compiler and interpreter systems convert English-like commands into executable machine or low-level language codes. Code-to-code translators are common in word processing software, supporting conversions between Microsoft Word, ASCII, and WordPerfect formats, for example.
As the term implies, grammar analysis systems are used to check spelling and grammar. For example, the grammar analysis facility in Microsoft Word for Office 97 continuously underlines spelling errors in red and grammatical errors in green as the user types. In addition to highlighting misspellings, commonly misused words, awkward sentence structures, awkward phrases, and incorrect punctuation, a sophisticated grammar analysis system can also provide substitutes for specific words, determine the reading level of a document, and provide status and statistical data for further analysis.
Record management systems read the contents of records (received, stored, and transmitted), analyze the contents, sort the records into proper categories, and add meaningful indexes or key words and phrases for future reference.
A natural language processing system can serve as a user interface to a database system, an expert system, or a specific application. A SQL command generator is a good example of a database system interface. Natural language interfaces show great promise for expert systems, and considerable research has already been done. Other natural language interfaces are used in data communications, manufacturing, and office automation.
Natural language processing will play an important role in future robotic systems. Robotics combines such features as speech recognition, natural language processing, natural language translation, image processing, and patte recognition, and is beyond the scope of this book.
Expert system (knowledge-based system) —
A computer program that emulates the thought process of a human expert.
Generator —
A routine that outputs one or more commands that the computer can execute.
Knowledge base —
A collection of data, algorithms, and heuristic rules that forms the core of an expert system.
Lexical analyzer —
A routine that performs semantic analysis, checking every word in a sentence against the correct spellings stored in the knowledge base and listing all the possible alteative meanings for the sentence.
Natural language processing shell —
A natural language processing user interface. The user communicates with the shell by entering plain English character strings. The shell translates the plain English strings into the appropriate commands and passes the commands to an application program.
Parse tree —
A hierarchical representation of words (conceptually similar to a diagrammed sentence) arranged in a form that allows a computer program to trace relationships and infer meanings.
Parser —
A routine that performs syntactic analysis, essentially diagramming a sentence to form a parse tree.
Semantic analysis —
A technique in which the system determines the meaning of each word by looking it up in a dictionary or a knowledge base.
Speech recognition —
An extension of natural language processing that uses a speech recognition routine (or a chip) to break continuous speech into a string of words, inputs the string to a natural language processing routine, and then passes the resulting commands to an application program.
Syntactic analysis —
A technique that allows a parser routine to, essentially, diagram a sentence to form a parse tree.
Word recognition —
The process of restructuring an input string into a series of noun phrases, verb phrases, prepositional phrases, adjective phrases, and so on.
Dragon Systems’ Naturally Speaking and IBM’s ViaVoice Gold are voice recognition software packages that might be used to support a speech recognition system. Other examples include Intellect from Artificial Intelligence Corp., RAMIS II English from Mathematica, Inc., Spock from Frey Associates, Inc., and NaturalLink from Texas Instruments.
1. Allen J., Natural Language Understanding, Benjamin/Cummings, Redwood City, CA, 1987.
2. Davis, W. S., Computers and Information Systems: An Introduction, West, Minneapolis, MN, 1997.
3. O’Shea, T. and Eisenstadt, M., Artificial Intelligence: Tools, Techniques, and Applications, Harper & Row, New York, 1984.
4. Rauch-Hindin, W. B., A Guide to Commercial Artificial Intelligence: Fundamentals and Real World Applications, Prentice-Hall, Englewood Cliffs, NJ, 1988.
5. Turban E., Expert Systems and Applied Artificial Intelligence, Macmillan, New York, 1992.
6. Williamson M., Artificial Intelligence for Microcomputers: The Guide for Business Decision Makers, Brady Communications, New York, 1986.
7. Winston, P. H. and Prendergast, K. A., The AI Business: Commercial Uses of Artificial Intelligence, MIT Press, Cambridge, MA, 1984.
اطلاعات کامپیوتری...برچسب : نویسنده : حسن eight8 بازدید : 217