What is grammatical structure of the word

3.6 More Radically Functionalist Schools

Grammatical structure is commonly assumed to exist in speakers’ minds. However, grammatical structure is also directly involved in social interaction in language use, and language use is central to accounting for language acquisition, language variation and language change. In the more dynamic process of language acquisition and language change, functional factors have been argued to play a role. In both language acquisition and language change, it has been argued that competing motivations among functional principles play a major role.

The concept of competing motivations is that (functional) principles may come into conflict such that there is no grammatical system that satisfies all of the functional principles. As a result, change occurs over time in acquisition and in the history of a language, and the languages of the world exhibit structural diversity even though their speakers’ linguistic behavior conforms to the same functional principles. (Competing motivation models do not presuppose that the competing principles are functional, and in fact competing motivation models are now used by many formalists; see below).

The most general and commonly offered example of competing motivations is that between economy and iconicity (see Linguistics: Iconicity). Economy is the principle that a speaker uses the least effort necessary to express himself or herself. For example, English leaves the singular number of nouns unexpressed: book-ϕ vs. book-s. Economy is considered to be a speaker-oriented functional principle: its influence on language is for the benefit of the speaker. Iconicity is, in part, the principle that all of the relevant parts of the meaning conveyed are in fact conveyed by grammatical elements in the utterance (words, inflections, etc.). For example, in some languages, both the singular and the plural of a noun are expressed by overt suffixes. This aspect of iconicity is considered to be hearer-oriented: any aspect of meaning left out by the speaker may not be recoverable by the hearer. Economy and iconicity compete with each other: a linguistic expression that is economical will not be iconic (since it leaves some elements of meaning unexpressed), and an expression that is iconic will not be economical (since certain elements of meaning are not left unexpressed). Hence, across languages, there is diversity in the expression of the category of number, and languages change from one form of expression of number to another over time.

Perhaps the best known competing motivations model in language acquisition is the competition model of E. Bates and B. MacWhinney (1989). In language change, a number of linguists have put forward competing motivations models, in particular J. Haiman (1985).

More recently, some functionalist linguists (P. Hopper, J. Haiman, and J. Bybee) have emphasized the dynamic character of language in ordinary use, and have argued that a speaker’s grammatical knowledge should not be considered to be as static and immutable as is usually believed. They argue that a speaker’s grammatical knowledge is not a tightly integrated system, but rather a more loosely structured inventory of conventionalized routines that have emerged through language use. The empirical research of the functionalist linguists who advocate this view has focused on the role of frequency of use on the entrenchment of grammatical knowledge, and on inductive models of abstracting grammatical knowledge from exposure to language use, both in acquisition and in adult usage. There have been a number of studies of inflectional paradigms of words that support the hypothesis that frequency of use influences grammatical representation; however, studies of syntactic construction in this approach are in their infancy.

The vocabulary, grammatical structure and usage conventions of any language are linked in innumerable ways with the social, cultural, and historical experience of its speakers. Drawing on examples from many languages, this article demonstrates the nature and range of these links. It highlights the culture-specific nature of many words, including terms for social categories, emotions, and value concepts. It explains the notion of cultural key words and the significance of lexical elaboration. In the domain of grammar, the article shows how grammatical constructions and marking may express and reflect culture-related meanings. In the realm of language in use, topics considered include discourse particles and interjections, linguistic routines, speech genres and speech styles, and broader discourse styles, demonstrating diverse ways in which language use can be related to differing cultural norms, values, and attitudes.

Abstract

Parsing is the task of analyzing grammatical structures of an input sentence and deriving its parse tree. Efficient solutions for parsing are needed in many applications such as natural language processing, bioinformatics, and pattern recognition. The Cocke–Younger–Kasami (CYK) algorithm is a well-known parsing algorithm that operates on context-free grammars in Chomsky normal form and has been extensively studied for execution on parallel machines. In this chapter, we analyze the parallelizing opportunities for the CYK algorithm and give an overview of existing implementations on different hardware architectures. We propose a novel, efficient streaming dataflow implementation of the CYK algorithm on reconfigurable hardware (Maxeler dataflow engines), which achieves 18–76 × speedup over an optimized sequential implementation for real-life grammars for natural language processing, depending on the length of the input string.

3.2.4 Encoding Meaning into Structure

The real problem lies in the structure of the grammar on page 92. It treats all the arithmetic operators in the same way, ignoring precedence. A rightmost derivation of

generates a different parse tree than does a leftmost derivation of the same string. The grammar is ambiguous.

The example in Fig. 3.1 showed a string with parentheses. The parentheses forced the leftmost and rightmost derivations into the same parse tree. That extra production in the grammar added a level to the parse tree that, in turn, forces the same evaluation order independent of derivation order.

In the classic expression grammar,

forms a level for + and -, forms alevel for × and , and forms a level for . The modified grammar derives a parse tree for that models standard algebraic precedence.

A postorder treewalk over this parse tree will first evaluate

and then add the result to . The grammar enforces the standard rules of arithmetic precedence. This grammar is unambiguous; the leftmost derivation produces the same parse tree.

The changes affect derivation lengths and parse tree sizes. The new nonterminals that enforce precedence add steps to the derivation and interior nodes to the tree. At the same time, moving the operators inline eliminated one production and one node per operator.

Other operations require high precedence. For example, array subscripts should be applied before standard arithmetic operations. This ensures, for example, that

evaluates to a value before adding it to , as opposed to treating as a subscript on some array whose location is computed as . Similarly, operations that change the type of a value, known as type casts in languages such as c or java, have higher precedence than arithmetic operations but lower precedence than parentheses or subscript operations.

If the language allows assignment inside expressions, the assignment operator should have low precedence. This ensures that the code completely evaluates both the left-hand side and the right-hand side of the assignment before performing the assignment. If assignment (←) had the same precedence as addition, for example, a left-to-right evaluation of

would assign ‘s value to rather than evaluating and then assigning that result to .

1.2 Assessing Structural Changes by Sampling along the Proficiency Continuum

By sampling the usage of speakers at various points along the proficiency continuum, ongoing changes in phonological and grammatical structure can be identified in an obsolescent language. Caution is necessary both in identifying and in interpreting evidence of change, however. In assessing change, the norms against which any deviation is measured must be established strictly in terms of the most conservative local usage. Local norms are essential, since any standardized or official form of the language may be historically irrelevant to the local form, based mainly on quite different dialects, for example, or relatively recently devised. Furthermore, structural changes found to be in progress are not necessarily the unequivocal result of obsolescence as such since intensive contact with another language often promotes structural change, whether language shift is underway or not. Reports of structural convergence appear, for example, both where a language long in contact with others has become obsolescent (as with Tariana, cited above) and where languages long in contact with one another all continue to be maintained (as in the Urdu, Marathi, and Kannada case discussed by Gumperz and Wilson 1971). Generally speaking, the broader the age-range encompassed by the proficiency continuum and the larger the number of speakers sampled at various points along it, the greater the likelihood that changes to which obsolescence contributes substantially can be identified. Where circumstances permit, longitudinal sampling can offer important additional evidence, by determining the degree to which innovative structural features once absent or rare among the oldest and most conservative speakers establish themselves subsequently among those who were previously the younger speakers.

Abstract:

This assignment facilitates the acquisition of etic perspectives of the target culture, as well as new vocabulary and new grammatical structures, through the use of mobile digital storytelling. Students are asked to explore a target culture through the lens of a nonnative of that culture, and then asked to put together a short narrated video on a given topic. Students can use a smartphone (or iPod Touch) to record video clips and then edit them together on their phone to create a longer video. Videos can then be uploaded to YouTube where fellow students will be able to view and comment on them. Although the original assignment was aimed at intermediate to advanced students in a language-learning classroom, the assignment can be utilized in other disciplines that deal with issues of group identity and self-identity.

3 STATE MODELS

If the scores used are log probabilities or log odds, then a finite state automaton is essentially a hidden Markov model (HMM), and these have been introduced recently into gene finding by several groups. The only fundamental difference from the dynamic programming schemes discussed above is that these models are fully probabilistic, which has certain advantages. One of the advantages is that the weighting problem is easier.

VEIL (Henderson et al., 1997) is an application of an HMM to the gene finding problem. In this model all the sensors are HMMs. The exon module is essentially a first-order inhomogeneous Markov chain, which is described above. This is the natural order for implementation in an HMM, because then each of the conditional probabilities of the inhomogeneous Markov chain corresponds to the probability of a transition from one state to the next in the HMM. It is not possible to avoid stop codons in the reading frame when using a first-order model, but in VEIL a few more states are added in a clever way, which makes the probability of a stop codon zero. Sensors for splice sites are made in a similar way. The individual modules are then combined essentially as in Figure 11.2 (i.e. frame consistency is not enforced). The combined model is one big HMM, and all the transitions have associated probabilities. These probabilities can be estimated from a set of training data by a maximum likelihood method. For combining the models this essentially boils down to counting occurrences of the different types of transitions in the dataset. Therefore, the implicit weighting of the individual sensors is not really an issue.

Although the way the optimal gene structure is found is similar in spirit to the dynamic programming above, it looks quite different in practice. This is because the dynamic programming is done at the level of the individual states in all the submodels; there are more than 200 such states in VEIL. Because the model is fully probabilistic, one can calculate the probability of any sequence of states for a given DNA sequence. This state sequence (called a path) determines the assignment of exons and introns. If the path goes through the exon model, that part of the sequence is labelled as exon; if it goes through the intron model it is labelled intron, and so forth. The dynamic programming algorithm, which is called the Viterbi algorithm, finds the most probable path through the model for a given sequence, and from this the predicted gene structure is derived (see Rabiner (1989) for a general introduction to HMMs).

This probabilistic model has the advantage of solving the problem of weighting the individual sensors. The maximum likelihood estimation of the parameters can be shown to be optimal if there are sufficient training data, and if the statistical nature of genes can be described by such a model. A weak part of VEIL is the first-order exon model, which is probably not capable of capturing the statistics of coding regions, and most other methods use fourth- or fifth-order models.

A HMM-based gene finder called HMMgene is currently being developed. The basic method is the same as VEIL, but it includes several extensions to the standard HMM methodology, which are described by Krogh (1997). One of the most important is that coding regions are modelled by a fourth-order inhomogeneous Markov chain instead of a first-order chain. This is done by an almost trivial extension of the standard HMM formalism, which allows a Markov chain of any order in a state of the model, whereas the standard HMM has a simple unconditional probability distribution over the four bases (corresponding to 0th order). The model is frame-aware and can predict any number of genes and partial genes, so the overall structure of the model is as in Figure 11.4 with transitions added to allow for begin and end in introns, as in Figure 11.3.

As already mentioned, the maximum likelihood estimation method works well if the model structure can describe the true statistics of genes. This is a very idealized assumption, and therefore HMMgene uses another method for estimating the parameters called conditional maximum likelihood (Juang and Rabiner, 1991; Krogh, 1994). Loosely speaking, maximum likelihood maximizes the probability of the DNA sequences in the training set, whereas conditional maximum likelihood maximizes the probability of the gene structures of these sequences, which, after all, is what we are interested in. This kind of optimization is conceptually similar to that used in GeneParser, where the prediction accuracy is also optimized. HMMgene also uses a dynamic programming algorithm different from the Viterbi algorithm for prediction of the gene structure. All of these methods have contributed to a high performance of HMMgene.

Genie is another example of a probabilistic state model which is called a generalized HMM (Kulp et al., 1996; Reese et al., 1997). Figure 11.4 is in fact Genie’s state structure, and both this figure and Figure 11.2 are essentially copied from Kulp et al. (1996). In Genie, the signal sensors (splice sites) and content sensors (coding potential) are neural networks, and the output of these networks is interpreted as probabilities. This interpretation requires estimation of additional probability parameters which work like weights on the sensors. So, although it is formulated as a probabilistic model, the weighting problem still appears in disguise. The algorithm for prediction is almost identical to the dynamic programming algorithm of the last section. A version of Genie also includes database similarities as part of the exon sensor (Kulp et al., 1997).

There are two main advantages of generalized HMMs compared with standard HMMs. First, the individual sensors can be of any type, such as neural networks, whereas in a standard HMM they are restricted by the HMM framework. Second, the length distribution (of, for example, coding regions) can be taken into account explicitly, whereas the natural length distribution for an HMM is a geometric distribution, which decays exponentially with the length. However, it is possible to have a fairly advanced length modelling in an HMM if several states are used (Durbin et al., 1998). The advantage of a system like HMMgene, on the other hand, is that it is one integrated model, which can be optimized all at once for maximum prediction accuracy.

Another gene finder based on a generalized HMM is GENSCAN (Burge and Karlin, 1997). The main differences between the GENSCAN state structure and that of Genie or HMMgene is that GENSCAN models the sequence in both directions simultaneously. In many gene finders, such as those described above, genes are first predicted on one strand, and then on the other. Modelling both strands simultaneously was done very successfully in GeneMark, and a similar method is implemented in GENSCAN. One advantage (and perhaps the main one) is that this construction avoids predictions of overlapping genes on the two strands, which presumably are very rare in the human genome. GENSCAN models any number of genes and partial genes like HMMgene. The sensors in GENSCAN are similar to those used in HMMgene. For instance, the coding sensor is a fifth-order inhomogeneous Markov chain. The signal sensors are essentially position-dependent weight matrices, and thus are also very similar to those of HMMgene, but there are more advanced features in the splice site models. GENSCAN also model promoters and the 5‘ and 3‘ UTRs.

2.2 The Professionalization of American Linguistics

By 1921, Sapir was able to draw on the analytic details of the American Indian languages on which he had worked to illustrate, in his book Language, the wide variety of grammatical structures represented in human speech. One of the most significant impacts of this highly successful book was to provide a model for the professionalization of academic linguistics in the US during the 1920s and early 1930s.

Under Sapir’s guidance, a distinctive American School of linguistics arose, focused on the empirical documentation of language, primarily in field situations. Although most of the students Sapir himself trained at Chicago and Yale worked largely if not exclusively on American Indian languages, the methods that were developed were transferable to other languages. In the mid-1930s Sapir directed a project to analyze English, and during World War II many of Sapir’s former students were recruited to use linguistic methods to develop teaching materials for such strategically important languages as Thai, Burmese, Mandarin, and Russian.

A distinction is sometimes drawn between the prewar generation of American linguists—dominated by Sapir and his students and emphasizing holistic descriptions of American Indian languages—and the immediate postwar generation, whose more rigid and focused formal methods were codified by Leonard Bloomfield and others. Since many of the major figures of ‘Bloomfieldian’ linguistics were Sapir’s students, this distinction is somewhat artificial, and a single American Structuralist tradition can be identified extending from the late 1920s through 1960 (Hymes and Fought 1981). There is little doubt that Sapir’s influence on this tradition was decisive.

3.5 Effects of Context

Sentences are normally not interpreted in isolation, which raises the question of how context can affect the processes by which they are understood. One line of research has emphasized the referential requirements of grammatical structures. One use of a modifier, such as a relative clause, is to select one referent from a set of possible referents. For instance, if there are two books on a table, you can ask for the ‘book that has the red cover.’ Some researchers (e.g., Altmann and Steedman 1988) have suggested that the difficulty of the relative clause in a ‘horse raced’ type garden-path arises simply because the sentence is presented out of context and there is no set of referents for the relative clause to select from. This suggestion entails that the garden-path would disappear in a context in which two horses are introduced, one of which was raced past a barn. Most experimental research fails to give strong support to this claim (cf. Mitchell 1994, Tanenhaus and Trueswell 1995). However, the claim may be correct for weaker garden-paths, for example the prepositional phrase attachment discussed above (‘The doctor examined the patient with a broken wrist’), at least when the verb does not require a prepositional phrase argument. And a related claim may even be correct for relative clause modification when temporal relations, not simply referential context, affect the plausibility of main verb vs. relative clause interpretations (cf. Tanenhaus and Trueswell 1995).

II.F Mathematical Linguistics

We start with a set W of basic units considered words. Mathematical linguistics is concerned with the formal theory of sentences, that is, sequences of words that are grammatically allowed and the grammatical structure of sentences (or longer units). This is syntax. Meaning (semantics) is usually not dealt with.

For a set X, let X^* be the set of finite sequences from X including the empty sequence e. For instance, if X is {0, 1}, then X^* is {e, 0, 1, 00, 01, 10, 11, 000 ⋯}. For a more important example, we can consider the family of all sequences of logical variables p, q, r, and ∨ (or), ∧ (and), (,) (parentheses), → (if then), ~ (not). The set of logical formulas will be a subset of this.

A phrase structure grammar is a quadruple (N, W, ρ, ψ), where (W set of words) is nonempty and finite, N (nonterminals) is a finite set disjoint from W, ψ ∈ N, and ρ is a finite subset of ((N ∪ W)^* W^*) × (N ∪ W)^*. The set N is a set of valid grammatical forms involving abstract concepts such as ψ (sentence) or subject, predicate, object. The set ρ (productions) is a set of ways we can substitute into a valid grammatical form to obtain another, more specific one. The element ψ is called the starting symbol. If (x, y) ∈ ρ, we are allowed to change any occurrence of x with y. Members of W are called terminals.

Consider logical formulas involving operations ∨, ~ and variables p, q, r. Let W = {p, q, r, ∨, (,), ~}, N = {ψ}. We derive formulas by successive substitution, as ψ,(ψ ∨ ψ), ((ψ ∨ ψ) ∨ ψ),((ψ ∨ ψ) ∨ ~ ψ), ((p ∨ ψ) ∨ ~ ψ),((p ∨ q) ∨ ~ ψ),((p ∨ q) ∨ ~ r).

An element y ∈ (N ∪ W)^* is said to be directly derived from x ∈ (N ∪ W)^* if x = azb, y = awb for some (z, w) ∈ ρ, a, b ∈ (N ∪ W)^*. An indirect derivation is a sequence of direct derivations. Here ρ = {(ψ, p),(ψ, q),(ψ, r),(ψ, ~ ψ), (ψ,(ψ ∨ ψ))}.

The language determined by a phrase structure grammar is the set of all a ∈ W^* that can be derived from ψ.

A grammar is called context free if and only if for all (a, b) ∈ ρ, a ∈ N, b ≠ e₀. This means that what items can be substituted for a given grammatical element do not depend on other grammatical elements. The grammar above is context free.

A grammar is called regular if for all (a, b) ∈ ρ we have a ∈ N, b = tn, where t ∈ W, n ∈ N, or n = e₀. This means at each derivation we go from t₁t₂ ⋯ t_rn to t₁t₂ ⋯ t_rt_r + 1m, where t_i are terminals, n, m nonterminals, (n, t_r + 1m) ∈ ρ. So we fill in one terminal at each step, going from left to right. The grammar mentioned above is not regular.

To recognize a grammar is to be able to tell whether or not a sequence from W^* is in the language. A grammar is regular if and only if some finite state machine recognizes it. The elements of W are input 1 at a time and outputs are “yes, no,” meaning all symbols up to the present are or are not in the language. Let the internal states of the machine be in 1–1 correspondence with all subsets of N, and let the initial state be ψ. For a set S₁ of nonterminals and input x let the next state be the set S₂ of all nonterminals z such that for some u ∈ S₁, (u, xz) is a production. Then at any time the state consists of all nonterminals that could occur after the given seqence of inputs. Let the output be “yes” if and only if for some u ∈ N,(u, x) ∈ ρ. This is if and only if the previous inputs together with the current input form a word in the language.

For the converse, if a finite state machine can recognize a language, let W be as in the language, N be the set of internal states, ψ the initial state, the productions the set of pairs (n₁, xn₂) such that if the machine is in state n₁ and x is input, state n₂ is the next state, and the set of pairs (n₁, x) such that in state n₁ after input x the machine answers “yes.”

A further characterization of regular language is the Myhill–Nerode theorem. Let W^* be considered a semigroup of words. Then a language L ⊂ W^* is regular if and only if the congruence {(x, y) ∈ W^* × W^* : axb ∈ L if and only if ayb ∈ L for all a, b ∈ L} has finitely many equivalence classes. This is if and only if there exists a finite semigroup H, a homomorphism h : W^* → H is onto, and a subset S ⊂ H such that L = h⁻¹(S).