Lab: Pretraining LMs - CLM

Introduction

In this lab, we will dive into the process of pretraining a Causal Language Model (CLM) from scratch using a subset of the Wiki dataset. Specifically, we will be training a small GPT-2 model. The goal of pretraining is to learn good representations of our data before fine-tuning the model on a specific task.

Causal Language Modeling, or Autoregressive Language Modeling, is a type of language modeling where the model makes predictions of the next token in the sequence, given the tokens that came before it. GPT-2 is a transformer model that was designed for this type of task.

The steps involved in pretraining a CLM are as follows:

1. Preparing the Environment: Setting up the necessary libraries and frameworks for training the model.

2. Preparing the Dataset: Downloading and preparing the Wiki dataset for our pretraining task.

3. Creating a Tokenizer: Training a tokenizer from scratch on our dataset.

4. Formatting the Data: Preparing our dataset in a format suitable for training our GPT-2 model.

5. Defining the Model: Initializing a GPT-2 model configuration.

6. Training the Model: Training our GPT-2 model on our prepared dataset.

7. Validating the Model: Checking the performance of our model on a holdout validation dataset.

8. Saving and Loading the Model: Storing our trained model for later use and loading it back into memory when needed.

By the end of this lab, you will have a working GPT-2 model that has been pretrained on a subset of the Wiki dataset, and a tokenizer that has been trained on the same data. This model will be ready to be fine-tuned on a downstream task of your choice.

In the next sections, we will delve into each of these steps in more detail.

Preparing the Environment

In this section, we will set up our working environment on Google Colab. We’ll need to install the appropriate libraries and dependencies required for pretraining our GPT-2 model.

Step 1: Update and Upgrade the System

First, we’ll update and upgrade the system to ensure that we have the latest software and libraries available.

!apt-get update
!apt-get upgrade -y

Step 2: Install Required Libraries

Next, we’ll need to install the required Python libraries, which include Hugging Face’s transformers and datasets libraries, and the tokenizers library.

!pip install transformers
!pip install datasets
!pip install tokenizers

The transformers library provides us with the implementation of the GPT-2 model, along with various utilities for working with transformer models.

The datasets library gives us access to a wide variety of datasets and also provides utilities for working with datasets in an efficient manner.

The tokenizers library is used for training our tokenizer from scratch.

Step 3: Import Necessary Libraries

Lastly, we’ll import the necessary libraries into our Python environment.

import torch
from torch.utils.data import Dataset, DataLoader
from transformers import (
    GPT2Tokenizer,
    GPT2LMHeadModel,
    TextDataset,
    DataCollatorForLanguageModeling,
)
from transformers import Trainer, TrainingArguments
from datasets import load_dataset
from tokenizers import Tokenizer, models, pre_tokenizers, decoders, processors

/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

With these in place, we are now ready to proceed to the next step, which is preparing our dataset for pretraining our model.

Remember that we are working with the Google Colab environment, which comes preloaded with many necessary libraries. However, depending on your specific needs, you may need to install or import additional libraries.

Preparing the Dataset

In this section, we will download and prepare a subset of the Wiki dataset for pretraining our GPT-2 model. The datasets library from Hugging Face provides an easy and efficient way to load the dataset.

Step 1: Download the Dataset

We will use the load_dataset function from the datasets library to download our dataset.

from datasets import load_dataset

# Load a subset of the Wikipedia dataset
dataset = load_dataset("wikipedia", "20220301.en", split="train[:10%]")

# Display the first few entries in the dataset
print(dataset[:5])

Found cached dataset wikipedia (/home/yj.lee/.cache/huggingface/datasets/wikipedia/20220301.en/2.0.0/aa542ed919df55cc5d3347f42dd4521d05ca68751f50dbc32bae2a7f1e167559)

{'id': ['12', '25', '39', '290', '303'], 'url': ['https://en.wikipedia.org/wiki/Anarchism', 'https://en.wikipedia.org/wiki/Autism', 'https://en.wikipedia.org/wiki/Albedo', 'https://en.wikipedia.org/wiki/A', 'https://en.wikipedia.org/wiki/Alabama'], 'title': ['Anarchism', 'Autism', 'Albedo', 'A', 'Alabama'], 'text': ['Anarchism is a political philosophy and movement that is sceptical of authority and rejects all involuntary, coercive forms of hierarchy. Anarchism calls for the abolition of the state, which it holds to be unnecessary, undesirable, and harmful. As a historically left-wing movement, placed on the farthest left of the political spectrum, it is usually described alongside communalism and libertarian Marxism as the libertarian wing (libertarian socialism) of the socialist movement, and has a strong historical association with anti-capitalism and socialism.\n\nHumans lived in societies without formal hierarchies long before the establishment of formal states, realms, or empires. With the rise of organised hierarchical bodies, scepticism toward authority also rose. Although traces of anarchist thought are found throughout history, modern anarchism emerged from the Enlightenment. During the latter half of the 19th and the first decades of the 20th century, the anarchist movement flourished in most parts of the world and had a significant role in workers\' struggles for emancipation. Various anarchist schools of thought formed during this period. Anarchists have taken part in several revolutions, most notably in the Paris Commune, the Russian Civil War and the Spanish Civil War, whose end marked the end of the classical era of anarchism. In the last decades of the 20th and into the 21st century, the anarchist movement has been resurgent once more.\n\nAnarchism employs a diversity of tactics in order to meet its ideal ends which can be broadly separated into revolutionary and evolutionary tactics; there is significant overlap between the two, which are merely descriptive. Revolutionary tactics aim to bring down authority and state, having taken a violent turn in the past, while evolutionary tactics aim to prefigure what an anarchist society would be like. Anarchist thought, criticism, and praxis have played a part in diverse areas of human society. Criticism of anarchism include claims that it is internally inconsistent, violent, or utopian.\n\nEtymology, terminology, and definition \n\nThe etymological origin of anarchism is from the Ancient Greek anarkhia, meaning "without a ruler", composed of the prefix an- ("without") and the word arkhos ("leader" or "ruler"). The suffix -ism denotes the ideological current that favours anarchy. Anarchism appears in English from 1642 as anarchisme and anarchy from 1539; early English usages emphasised a sense of disorder. Various factions within the French Revolution labelled their opponents as anarchists, although few such accused shared many views with later anarchists. Many revolutionaries of the 19th century such as William Godwin (1756–1836) and Wilhelm Weitling (1808–1871) would contribute to the anarchist doctrines of the next generation but did not use anarchist or anarchism in describing themselves or their beliefs.\n\nThe first political philosopher to call himself an anarchist () was Pierre-Joseph Proudhon (1809–1865), marking the formal birth of anarchism in the mid-19th century. Since the 1890s and beginning in France, libertarianism has often been used as a synonym for anarchism and its use as a synonym is still common outside the United States. Some usages of libertarianism refer to individualistic free-market philosophy only, and free-market anarchism in particular is termed libertarian anarchism.\n\nWhile the term libertarian has been largely synonymous with anarchism, its meaning has more recently diluted with wider adoption from ideologically disparate groups, including both the New Left and libertarian Marxists, who do not associate themselves with authoritarian socialists or a vanguard party, and extreme cultural liberals, who are primarily concerned with civil liberties. Additionally, some anarchists use libertarian socialist to avoid anarchism\'s negative connotations and emphasise its connections with socialism. Anarchism is broadly used to describe the anti-authoritarian wing of the socialist movement. Anarchism is contrasted to socialist forms which are state-oriented or from above. Scholars of anarchism generally highlight anarchism\'s socialist credentials and criticise attempts at creating dichotomies between the two. Some scholars describe anarchism as having many influences from liberalism, and being both liberals and socialists but more so, while most scholars reject anarcho-capitalism as a misunderstanding of anarchist principles.\n\nWhile opposition to the state is central to anarchist thought, defining anarchism is not an easy task for scholars, as there is a lot of discussion among scholars and anarchists on the matter, and various currents perceive anarchism slightly differently. Major definitional elements include the will for a non-coercive society, the rejection of the state apparatus, the belief that human nature allows humans to exist in or progress toward such a non-coercive society, and a suggestion on how to act to pursue the ideal of anarchy.\n\nHistory\n\nPre-modern era \n\nBefore the establishment of towns and cities, an established authority did not exist. It was after the creation of institutions of authority that anarchistic ideas espoused as a reaction. The most notable precursors to anarchism in the ancient world were in China and Greece. In China, philosophical anarchism (the discussion on the legitimacy of the state) was delineated by Taoist philosophers Zhuang Zhou and Laozi. Alongside Stoicism, Taoism has been said to have had "significant anticipations" of anarchism.\n \nAnarchic attitudes were also articulated by tragedians and philosophers in Greece. Aeschylus and Sophocles used the myth of Antigone to illustrate the conflict between rules set by the state and personal autonomy. Socrates questioned Athenian authorities constantly and insisted on the right of individual freedom of conscience. Cynics dismissed human law (nomos) and associated authorities while trying to live according to nature (physis). Stoics were supportive of a society based on unofficial and friendly relations among its citizens without the presence of a state.\n\nIn medieval Europe, there was no anarchistic activity except some ascetic religious movements. These, and other Muslim movements, later gave birth to religious anarchism. In the Sasanian Empire, Mazdak called for an egalitarian society and the abolition of monarchy, only to be soon executed by Emperor Kavad I.\n\nIn Basra, religious sects preached against the state. In Europe, various sects developed anti-state and libertarian tendencies. Renewed interest in antiquity during the Renaissance and in private judgment during the Reformation restored elements of anti-authoritarian secularism, particularly in France. Enlightenment challenges to intellectual authority (secular and religious) and the revolutions of the 1790s and 1848 all spurred the ideological development of what became the era of classical anarchism.\n\nModern era \nDuring the French Revolution, partisan groups such as the Enragés and the  saw a turning point in the fermentation of anti-state and federalist sentiments. The first anarchist currents developed throughout the 18th century as William Godwin espoused philosophical anarchism in England, morally delegitimising the state, Max Stirner\'s thinking paved the way to individualism and Pierre-Joseph Proudhon\'s theory of mutualism found fertile soil in France. By the late 1870s, various anarchist schools of thought had become well-defined and a wave of then unprecedented globalisation occurred from 1880 to 1914. This era of classical anarchism lasted until the end of the Spanish Civil War and is considered the golden age of anarchism.\n\nDrawing from mutualism, Mikhail Bakunin founded collectivist anarchism and entered the International Workingmen\'s Association, a class worker union later known as the First International that formed in 1864 to unite diverse revolutionary currents. The International became a significant political force, with Karl Marx being a leading figure and a member of its General Council. Bakunin\'s faction (the Jura Federation) and Proudhon\'s followers (the mutualists) opposed state socialism, advocating political abstentionism and small property holdings. After bitter disputes, the Bakuninists were expelled from the International by the Marxists at the 1872 Hague Congress. Anarchists were treated similarly in the Second International, being ultimately expelled in 1896. Bakunin famously predicted that if revolutionaries gained power by Marx\'s terms, they would end up the new tyrants of workers. In response to their expulsion from the First International, anarchists formed the St. Imier International. Under the influence of Peter Kropotkin, a Russian philosopher and scientist, anarcho-communism overlapped with collectivism. Anarcho-communists, who drew inspiration from the 1871 Paris Commune, advocated for free federation and for the distribution of goods according to one\'s needs.\n\nAt the turn of the century, anarchism had spread all over the world. It was a notable feature of the international syndicalism movement. In China, small groups of students imported the humanistic pro-science version of anarcho-communism. Tokyo was a hotspot for rebellious youth from countries of the far east, travelling to the Japanese capital to study. In Latin America, Argentina was a stronghold for anarcho-syndicalism, where it became the most prominent left-wing ideology. During this time, a minority of anarchists adopted tactics of revolutionary political violence. This strategy became known as propaganda of the deed. The dismemberment of the French socialist movement into many groups and the execution and exile of many Communards to penal colonies following the suppression of the Paris Commune favoured individualist political expression and acts. Even though many anarchists distanced themselves from these terrorist acts, infamy came upon the movement and attempts were made to exclude them from American immigration, including the Immigration Act of 1903, also called the Anarchist Exclusion Act. Illegalism was another strategy which some anarchists adopted during this period.\n\nDespite concerns, anarchists enthusiastically participated in the Russian Revolution in opposition to the White movement; however, they met harsh suppression after the Bolshevik government was stabilised. Several anarchists from Petrograd and Moscow fled to Ukraine, notably leading to the Kronstadt rebellion and Nestor Makhno\'s struggle in the Free Territory. With the anarchists being crushed in Russia, two new antithetical currents emerged, namely platformism and synthesis anarchism. The former sought to create a coherent group that would push for revolution while the latter were against anything that would resemble a political party. Seeing the victories of the Bolsheviks in the October Revolution and the resulting Russian Civil War, many workers and activists turned to communist parties which grew at the expense of anarchism and other socialist movements. In France and the United States, members of major syndicalist movements such as the General Confederation of Labour and the Industrial Workers of the World left their organisations and joined the Communist International.\n\nIn the Spanish Civil War of 1936, anarchists and syndicalists (CNT and FAI) once again allied themselves with various currents of leftists. A long tradition of Spanish anarchism led to anarchists playing a pivotal role in the war. In response to the army rebellion, an anarchist-inspired movement of peasants and workers, supported by armed militias, took control of Barcelona and of large areas of rural Spain, where they collectivised the land. The Soviet Union provided some limited assistance at the beginning of the war, but the result was a bitter fight among communists and anarchists at a series of events named May Days as Joseph Stalin tried to seize control of the Republicans.\n\nPost-war era \n\nAt the end of World War II, the anarchist movement was severely weakened. The 1960s witnessed a revival of anarchism, likely caused by a perceived failure of Marxism–Leninism and tensions built by the Cold War. During this time, anarchism found a presence in other movements critical towards both capitalism and the state such as the anti-nuclear, environmental, and peace movements, the counterculture of the 1960s, and the New Left. It also saw a transition from its previous revolutionary nature to provocative anti-capitalist reformism. Anarchism became associated with punk subculture as exemplified by bands such as Crass and the Sex Pistols. The established feminist tendencies of anarcha-feminism returned with vigour during the second wave of feminism. Black anarchism began to take form at this time and influenced anarchism\'s move from a Eurocentric demographic. This coincided with its failure to gain traction in Northern Europe and its unprecedented height in Latin America.\n\nAround the turn of the 21st century, anarchism grew in popularity and influence within anti-capitalist, anti-war and anti-globalisation movements. Anarchists became known for their involvement in protests against the World Trade Organization (WTO), the Group of Eight and the World Economic Forum. During the protests, ad hoc leaderless anonymous cadres known as black blocs engaged in rioting, property destruction and violent confrontations with the police. Other organisational tactics pioneered in this time include affinity groups, security culture and the use of decentralised technologies such as the Internet. A significant event of this period was the confrontations at the 1999 Seattle WTO conference. Anarchist ideas have been influential in the development of the Zapatistas in Mexico and the Democratic Federation of Northern Syria, more commonly known as Rojava, a de facto autonomous region in northern Syria.\n\nThought \n\nAnarchist schools of thought have been generally grouped into two main historical traditions, social anarchism and individualist anarchism, owing to their different origins, values and evolution. The individualist current emphasises negative liberty in opposing restraints upon the free individual, while the social current emphasises positive liberty in aiming to achieve the free potential of society through equality and social ownership. In a chronological sense, anarchism can be segmented by the classical currents of the late 19th century and the post-classical currents (anarcha-feminism, green anarchism, and post-anarchism) developed thereafter.\n\nBeyond the specific factions of anarchist movements which constitute political anarchism lies philosophical anarchism which holds that the state lacks moral legitimacy, without necessarily accepting the imperative of revolution to eliminate it. A component especially of individualist anarchism, philosophical anarchism may tolerate the existence of a minimal state but claims that citizens have no moral obligation to obey government when it conflicts with individual autonomy. Anarchism pays significant attention to moral arguments since ethics have a central role in anarchist philosophy. Anarchism\'s emphasis on anti-capitalism, egalitarianism, and for the extension of community and individuality sets it apart from anarcho-capitalism and other types of economic libertarianism.\n\nAnarchism is usually placed on the far-left of the political spectrum. Much of its economics and legal philosophy reflect anti-authoritarian, anti-statist, libertarian, and radical interpretations of left-wing and socialist politics such as collectivism, communism, individualism, mutualism, and syndicalism, among other libertarian socialist economic theories. As anarchism does not offer a fixed body of doctrine from a single particular worldview, many anarchist types and traditions exist and varieties of anarchy diverge widely. One reaction against sectarianism within the anarchist milieu was anarchism without adjectives, a call for toleration and unity among anarchists first adopted by Fernando Tarrida del Mármol in 1889 in response to the bitter debates of anarchist theory at the time. Belief in political nihilism has been espoused by anarchists. Despite separation, the various anarchist schools of thought are not seen as distinct entities but rather as tendencies that intermingle and are connected through a set of uniform principles such as individual and local autonomy, mutual aid, network organisation, communal democracy, justified authority and decentralisation.\n\nClassical \n\nInceptive currents among classical anarchist currents were mutualism and individualism. They were followed by the major currents of social anarchism (collectivist, communist and syndicalist). They differ on organisational and economic aspects of their ideal society.\n\nMutualism is an 18th-century economic theory that was developed into anarchist theory by Pierre-Joseph Proudhon. Its aims include reciprocity, free association, voluntary contract, federation and monetary reform of both credit and currency that would be regulated by a bank of the people. Mutualism has been retrospectively characterised as ideologically situated between individualist and collectivist forms of anarchism. In What Is Property? (1840), Proudhon first characterised his goal as a "third form of society, the synthesis of communism and property." Collectivist anarchism is a revolutionary socialist form of anarchism commonly associated with Mikhail Bakunin. Collectivist anarchists advocate collective ownership of the means of production which is theorised to be achieved through violent revolution and that workers be paid according to time worked, rather than goods being distributed according to need as in communism. Collectivist anarchism arose alongside Marxism but rejected the dictatorship of the proletariat despite the stated Marxist goal of a collectivist stateless society.\n\nAnarcho-communism is a theory of anarchism that advocates a communist society with common ownership of the means of production, direct democracy and a horizontal network of voluntary associations, workers\' councils and worker cooperatives, with production and consumption based on the guiding principle "From each according to his ability, to each according to his need." Anarcho-communism developed from radical socialist currents after the French Revolution but was first formulated as such in the Italian section of the First International. It was later expanded upon in the theoretical work of Peter Kropotkin, whose specific style would go onto become the dominating view of anarchists by the late 19th century. Anarcho-syndicalism is a branch of anarchism that views labour syndicates as a potential force for revolutionary social change, replacing capitalism and the state with a new society democratically self-managed by workers. The basic principles of anarcho-syndicalism are direct action, workers\' solidarity and workers\' self-management.\n\nIndividualist anarchism is a set of several traditions of thought within the anarchist movement that emphasise the individual and their will over any kinds of external determinants. Early influences on individualist forms of anarchism include William Godwin, Max Stirner, and Henry David Thoreau. Through many countries, individualist anarchism attracted a small yet diverse following of Bohemian artists and intellectuals as well as young anarchist outlaws in what became known as illegalism and individual reclamation.\n\nPost-classical and contemporary \n\nAnarchist principles undergird contemporary radical social movements of the left. Interest in the anarchist movement developed alongside momentum in the anti-globalisation movement, whose leading activist networks were anarchist in orientation. As the movement shaped 21st century radicalism, wider embrace of anarchist principles signaled a revival of interest. Anarchism has continued to generate many philosophies and movements, at times eclectic, drawing upon various sources and combining disparate concepts to create new philosophical approaches. The anti-capitalist tradition of classical anarchism has remained prominent within contemporary currents.\n\nContemporary news coverage which emphasizes black bloc demonstrations has reinforced anarchism\'s historical association with chaos and violence. Its publicity has also led more scholars in fields such as anthropology and history to engage with the anarchist movement, although contemporary anarchism favours actions over academic theory. Various anarchist groups, tendencies, and schools of thought exist today, making it difficult to describe the contemporary anarchist movement. While theorists and activists have established "relatively stable constellations of anarchist principles", there is no consensus on which principles are core and commentators describe multiple anarchisms, rather than a singular anarchism, in which common principles are shared between schools of anarchism while each group prioritizes those principles differently. Gender equality can be a common principle, although it ranks as a higher priority to anarcha-feminists than anarcho-communists.\n\nAnarchists are generally committed against coercive authority in all forms, namely "all centralized and hierarchical forms of government (e.g., monarchy, representative democracy, state socialism, etc.), economic class systems (e.g., capitalism, Bolshevism, feudalism, slavery, etc.), autocratic religions (e.g., fundamentalist Islam, Roman Catholicism, etc.), patriarchy, heterosexism, white supremacy, and imperialism." Anarchist schools disagree on the methods by which these forms should be opposed. The principle of equal liberty is closer to anarchist political ethics in that it transcends both the liberal and socialist traditions. This entails that liberty and equality cannot be implemented within the state, resulting in the questioning of all forms of domination and hierarchy.\n\nTactics \nAnarchists\' tactics take various forms but in general serve two major goals, namely to first oppose the Establishment and secondly to promote anarchist ethics and reflect an anarchist vision of society, illustrating the unity of means and ends. A broad categorisation can be made between aims to destroy oppressive states and institutions by revolutionary means on one hand and aims to change society through evolutionary means on the other. Evolutionary tactics embrace nonviolence, reject violence and take a gradual approach to anarchist aims, although there is significant overlap between the two.\n\nAnarchist tactics have shifted during the course of the last century. Anarchists during the early 20th century focused more on strikes and militancy while contemporary anarchists use a broader array of approaches.\n\nClassical era tactics \n\nDuring the classical era, anarchists had a militant tendency. Not only did they confront state armed forces, as in Spain and Ukraine, but some of them also employed terrorism as propaganda of the deed. Assassination attempts were carried out against heads of state, some of which were successful. Anarchists also took part in revolutions. Many anarchists, especially the Galleanists, believed that these attempts would be the impetus for a revolution against capitalism and the state. Many of these attacks were done by individual assailants and the majority took place in the late 1870s, the early 1880s and the 1890s, with some still occurring in the early 1900s. Their decrease in prevalence was the result of further judicial power and targeting and cataloging by state institutions.\n\nAnarchist perspectives towards violence have always been controversial. Anarcho-pacifists advocate for non-violence means to achieve their stateless, nonviolent ends. Other anarchist groups advocate direct action, a tactic which can include acts of sabotage or terrorism. This attitude was quite prominent a century ago when seeing the state as a tyrant and some anarchists believing that they had every right to oppose its oppression by any means possible. Emma Goldman and Errico Malatesta, who were proponents of limited use of violence, stated that violence is merely a reaction to state violence as a necessary evil.\n\nAnarchists took an active role in strike actions, although they tended to be antipathetic to formal syndicalism, seeing it as reformist. They saw it as a part of the movement which sought to overthrow the state and capitalism. Anarchists also reinforced their propaganda within the arts, some of whom practiced naturism and nudism. Those anarchists also built communities which were based on friendship and were involved in the news media.\n\nRevolutionary tactics \n\nIn the current era, Italian anarchist Alfredo Bonanno, a proponent of insurrectionary anarchism, has reinstated the debate on violence by rejecting the nonviolence tactic adopted since the late 19th century by Kropotkin and other prominent anarchists afterwards. Both Bonanno and the French group The Invisible Committee advocate for small, informal affiliation groups, where each member is responsible for their own actions but works together to bring down oppression utilizing sabotage and other violent means against state, capitalism, and other enemies. Members of The Invisible Committee were arrested in 2008 on various charges, terrorism included.\n\nOverall, contemporary anarchists are much less violent and militant than their ideological ancestors. They mostly engage in confronting the police during demonstrations and riots, especially in countries such as Canada, Greece, and Mexico. Militant black bloc protest groups are known for clashing with the police; however, anarchists not only clash with state operators, they also engage in the struggle against fascists and racists, taking anti-fascist action and mobilizing to prevent hate rallies from happening.\n\nEvolutionary tactics \nAnarchists commonly employ direct action. This can take the form of disrupting and protesting against unjust hierarchy, or the form of self-managing their lives through the creation of counter-institutions such as communes and non-hierarchical collectives. Decision-making is often handled in an anti-authoritarian way, with everyone having equal say in each decision, an approach known as horizontalism. Contemporary-era anarchists have been engaging with various grassroots movements that are more or less based on horizontalism, although not explicitly anarchist, respecting personal autonomy and participating in mass activism such as strikes and demonstrations. In contrast with the big-A anarchism of the classical era, the newly coined term small-a anarchism signals their tendency not to base their thoughts and actions on classical-era anarchism or to refer to classical anarchists such as Peter Kropotkin and Pierre-Joseph Proudhon to justify their opinions. Those anarchists would rather base their thought and praxis on their own experience which they will later theorize.\n\nThe decision-making process of small anarchist affinity groups plays a significant tactical role. Anarchists have employed various methods in order to build a rough consensus among members of their group without the need of a leader or a leading group. One way is for an individual from the group to play the role of facilitator to help achieve a consensus without taking part in the discussion themselves or promoting a specific point. Minorities usually accept rough consensus, except when they feel the proposal contradicts anarchist ethics, goals and values. Anarchists usually form small groups (5–20 individuals) to enhance autonomy and friendships among their members. These kinds of groups more often than not interconnect with each other, forming larger networks. Anarchists still support and participate in strikes, especially wildcat strikes as these are leaderless strikes not organised centrally by a syndicate.\n\nAs in the past, newspapers and journals are used, and anarchists have gone online in the World Wide Web to spread their message. Anarchists have found it easier to create websites because of distributional and other difficulties, hosting electronic libraries and other portals. Anarchists were also involved in developing various software that are available for free. The way these hacktivists work to develop and distribute resembles the anarchist ideals, especially when it comes to preserving users\' privacy from state surveillance.\n\nAnarchists organize themselves to squat and reclaim public spaces. During important events such as protests and when spaces are being occupied, they are often called Temporary Autonomous Zones (TAZ), spaces where art, poetry, and surrealism are blended to display the anarchist ideal. As seen by anarchists, squatting is a way to regain urban space from the capitalist market, serving pragmatical needs and also being an exemplary direct action. Acquiring space enables anarchists to experiment with their ideas and build social bonds. Adding up these tactics while having in mind that not all anarchists share the same attitudes towards them, along with various forms of protesting at highly symbolic events, make up a carnivalesque atmosphere that is part of contemporary anarchist vividity.\n\nKey issues \n\nAs anarchism is a philosophy that embodies many diverse attitudes, tendencies, and schools of thought; disagreement over questions of values, ideology, and tactics is common. Its diversity has led to widely different uses of identical terms among different anarchist traditions which has created a number of definitional concerns in anarchist theory. The compatibility of capitalism, nationalism, and religion with anarchism is widely disputed, and anarchism enjoys complex relationships with ideologies such as communism, collectivism, Marxism, and trade unionism. Anarchists may be motivated by humanism, divine authority, enlightened self-interest, veganism, or any number of alternative ethical doctrines. Phenomena such as civilisation, technology (e.g. within anarcho-primitivism), and the democratic process may be sharply criticised within some anarchist tendencies and simultaneously lauded in others.\n\nGender, sexuality, and free love \n\nAs gender and sexuality carry along them dynamics of hierarchy, many anarchists address, analyse, and oppose the suppression of one\'s autonomy imposed by gender roles.\n\nSexuality was not often discussed by classical anarchists but the few that did felt that an anarchist society would lead to sexuality naturally developing. Sexual violence was a concern for anarchists such as Benjamin Tucker, who opposed age of consent laws, believing they would benefit predatory men. A historical current that arose and flourished during 1890 and 1920 within anarchism was free love. In contemporary anarchism, this current survives as a tendency to support polyamory and queer anarchism. Free love advocates were against marriage, which they saw as a way of men imposing authority over women, largely because marriage law greatly favoured the power of men. The notion of free love was much broader and included a critique of the established order that limited women\'s sexual freedom and pleasure. Those free love movements contributed to the establishment of communal houses, where large groups of travelers, anarchists and other activists slept in beds together. Free love had roots both in Europe and the United States; however, some anarchists struggled with the jealousy that arose from free love. Anarchist feminists were advocates of free love, against marriage, and pro-choice (utilising a contemporary term), and had a similar agenda. Anarchist and non-anarchist feminists differed on suffrage but were supportive of one another.\n\nDuring the second half of the 20th century, anarchism intermingled with the second wave of feminism, radicalising some currents of the feminist movement and being influenced as well. By the latest decades of the 20th century, anarchists and feminists were advocating for the rights and autonomy of women, gays, queers and other marginalised groups, with some feminist thinkers suggesting a fusion of the two currents. With the third wave of feminism, sexual identity and compulsory heterosexuality became a subject of study for anarchists, yielding a post-structuralist critique of sexual normality. Some anarchists distanced themselves from this line of thinking, suggesting that it leaned towards an individualism that was dropping the cause of social liberation.\n\nAnarchism and education \n\nThe interest of anarchists in education stretches back to the first emergence of classical anarchism. Anarchists consider proper education, one which sets the foundations of the future autonomy of the individual and the society, to be an act of mutual aid. Anarchist writers such as William Godwin (Political Justice) and Max Stirner ("The False Principle of Our Education") attacked both state education and private education as another means by which the ruling class replicate their privileges.\n\nIn 1901, Catalan anarchist and free thinker Francisco Ferrer established the Escuela Moderna in Barcelona as an opposition to the established education system which was dictated largely by the Catholic Church. Ferrer\'s approach was secular, rejecting both state and church involvement in the educational process whilst giving pupils large amounts of autonomy in planning their work and attendance. Ferrer aimed to educate the working class and explicitly sought to foster class consciousness among students. The school closed after constant harassment by the state and Ferrer was later arrested. Nonetheless, his ideas formed the inspiration for a series of modern schools around the world. Christian anarchist Leo Tolstoy, who published the essay Education and Culture, also established a similar school with its founding principle being that "for education to be effective it had to be free." In a similar token, A. S. Neill founded what became the Summerhill School in 1921, also declaring being free from coercion.\n\nAnarchist education is based largely on the idea that a child\'s right to develop freely and without manipulation ought to be respected and that rationality would lead children to morally good conclusions; however, there has been little consensus among anarchist figures as to what constitutes manipulation. Ferrer believed that moral indoctrination was necessary and explicitly taught pupils that equality, liberty and social justice were not possible under capitalism, along with other critiques of government and nationalism.\n\nLate 20th century and contemporary anarchist writers (Paul Goodman, Herbert Read, and Colin Ward) intensified and expanded the anarchist critique of state education, largely focusing on the need for a system that focuses on children\'s creativity rather than on their ability to attain a career or participate in consumerism as part of a consumer society. Contemporary anarchists such as Ward claim that state education serves to perpetuate socioeconomic inequality.\n\nWhile few anarchist education institutions have survived to the modern-day, major tenets of anarchist schools, among them respect for child autonomy and relying on reasoning rather than indoctrination as a teaching method, have spread among mainstream educational institutions. Judith Suissa names three schools as explicitly anarchists schools, namely the Free Skool Santa Cruz in the United States which is part of a wider American-Canadian network of schools, the Self-Managed Learning College in Brighton, England, and the Paideia School in Spain.\n\nAnarchism and the state \n\nObjection to the state and its institutions is a sine qua non of anarchism. Anarchists consider the state as a tool of domination and believe it to be illegitimate regardless of its political tendencies. Instead of people being able to control the aspects of their life, major decisions are taken by a small elite. Authority ultimately rests solely on power, regardless of whether that power is open or transparent, as it still has the ability to coerce people. Another anarchist argument against states is that the people constituting a government, even the most altruistic among officials, will unavoidably seek to gain more power, leading to corruption. Anarchists consider the idea that the state is the collective will of the people to be an unachievable fiction due to the fact that the ruling class is distinct from the rest of society.\n\nSpecific anarchist attitudes towards the state vary. Robert Paul Wolff believed that the tension between authority and autonomy would mean the state could never be legitimate. Bakunin saw the state as meaning "coercion, domination by means of coercion, camouflaged if possible but unceremonious and overt if need be." A. John Simmons and Leslie Green, who leaned toward philosophical anarchism, believed that the state could be legitimate if it is governed by consensus, although they saw this as highly unlikely. Beliefs on how to abolish the state also differ.\n\nAnarchism and the arts \n\nThe connection between anarchism and art was quite profound during the classical era of anarchism, especially among artistic currents that were developing during that era such as futurists, surrealists and others. In literature, anarchism was mostly associated with the New Apocalyptics and the neo-romanticism movement. In music, anarchism has been associated with music scenes such as punk. Anarchists such as Leo Tolstoy and Herbert Read stated that the border between the artist and the non-artist, what separates art from a daily act, is a construct produced by the alienation caused by capitalism and it prevents humans from living a joyful life.\n\nOther anarchists advocated for or used art as a means to achieve anarchist ends. In his book Breaking the Spell: A History of Anarchist Filmmakers, Videotape Guerrillas, and Digital Ninjas, Chris Robé claims that "anarchist-inflected practices have increasingly structured movement-based video activism." Throughout the 20th century, many prominent anarchists (Peter Kropotkin, Emma Goldman, Gustav Landauer and Camillo Berneri) and publications such as Anarchy wrote about matters pertaining to the arts.\n\nThree overlapping properties made art useful to anarchists. It could depict a critique of existing society and hierarchies, serve as a prefigurative tool to reflect the anarchist ideal society and even turn into a means of direct action such as in protests. As it appeals to both emotion and reason, art could appeal to the whole human and have a powerful effect. The 19th-century neo-impressionist movement had an ecological aesthetic and offered an example of an anarchist perception of the road towards socialism. In Les chataigniers a Osny by anarchist painter Camille Pissarro, the blending of aesthetic and social harmony is prefiguring an ideal anarchistic agrarian community.\n\nAnalysis \nThe most common critique of anarchism is that humans cannot self-govern and so a state is necessary for human survival. Philosopher Bertrand Russell supported this critique, stating that "[p]eace and war, tariffs, regulations of sanitary conditions and the sale of noxious drugs, the preservation of a just system of distribution: these, among others, are functions which could hardly be performed in a community in which there was no central government." Another common criticism of anarchism is that it fits a world of isolation in which only the small enough entities can be self-governing; a response would be that major anarchist thinkers advocated anarchist federalism.\n\nPhilosophy lecturer Andrew G. Fiala composed a list of common arguments against anarchism which includes critiques such as that anarchism is innately related to violence and destruction, not only in the pragmatic world, such as at protests, but in the world of ethics as well. Secondly, anarchism is evaluated as unfeasible or utopian since the state cannot be defeated practically. This line of arguments most often calls for political action within the system to reform it. The third argument is that anarchism is self-contradictory. While it advocates for no-one to archiei, if accepted by the many, then anarchism would turn into the ruling political theory. In this line of criticism also comes the self-contradiction that anarchism calls for collective action whilst endorsing the autonomy of the individual, hence no collective action can be taken. Lastly, Fiala mentions a critique towards philosophical anarchism of being ineffective (all talk and thoughts) and in the meantime capitalism and bourgeois class remains strong.\n\nPhilosophical anarchism has met the criticism of members of academia following the release of pro-anarchist books such as A. John Simmons\' Moral Principles and Political Obligations. Law professor William A. Edmundson authored an essay to argue against three major philosophical anarchist principles which he finds fallacious. Edmundson says that while the individual does not owe the state a duty of obedience, this does not imply that anarchism is the inevitable conclusion and the state is still morally legitimate. In The Problem of Political Authority, Michael Huemer defends philosophical anarchism, claiming that "political authority is a moral illusion."\n\nOne of the earliest criticisms is that anarchism defies and fails to understand the biological inclination to authority. Joseph Raz states that the acceptance of authority implies the belief that following their instructions will afford more success. Raz believes that this argument is true in following both authorities\' successful and mistaken instruction. Anarchists reject this criticism because challenging or disobeying authority does not entail the disappearance of its advantages by acknowledging authority such as doctors or lawyers as reliable, nor does it involve a complete surrender of independent judgment. Anarchist perception of human nature, rejection of the state, and commitment to social revolution has been criticised by academics as naive, overly simplistic, and unrealistic, respectively. Classical anarchism has been criticised for relying too heavily on the belief that the abolition of the state will lead to human cooperation prospering.\n\nFriedrich Engels, considered to be one of the principal founders of Marxism, criticised anarchism\'s anti-authoritarianism as inherently counter-revolutionary because in his view a revolution is by itself authoritarian. Academic John Molyneux writes in his book Anarchism: A Marxist Criticism that "anarchism cannot win", believing that it lacks the ability to properly implement its ideas. The Marxist criticism of anarchism is that it has a utopian character because all individuals should have anarchist views and values. According to the Marxist view, that a social idea would follow directly from this human ideal and out of the free will of every individual formed its essence. Marxists state that this contradiction was responsible for their inability to act. In the anarchist vision, the conflict between liberty and equality was resolved through coexistence and intertwining.\n\nSee also \n\n Anarchism by country\n Governance without government\n List of anarchist political ideologies\n List of books about anarchism\n\nReferences\n\nCitations\n\nNotes\n\nSources\n\nPrimary sources\n\nSecondary sources\n\nTertiary sources\n\nFurther reading \n \n  Criticism of philosophical anarchism.\n \n  A defence of philosophical anarchism, stating that "both kinds of \'anarchism\' [i.e. philosophical and political anarchism] are philosophical and political claims." (p.\xa0137)\n  Anarchistic popular fiction novel.\n \n \n \n  An argument for philosophical anarchism.\n\nExternal links \n Anarchy Archives. Anarchy Archives is an online research center on the history and theory of anarchism.\n\n \nAnti-capitalism\nAnti-fascism\nEconomic ideologies\nLeft-wing politics\nLibertarian socialism\nLibertarianism\nPolitical culture\nPolitical movements\nPolitical ideologies\nSocial theories\nSocialism\nFar-left politics', 'Autism is a neurodevelopmental disorder characterized by difficulties with social interaction and communication, and by restricted and repetitive behavior. Parents often notice signs during the first three years of their child\'s life. These signs often develop gradually, though some autistic children experience regression in their communication and social skills after reaching developmental milestones at a normal pace.\n\nAutism is associated with a combination of genetic and environmental factors. Risk factors during pregnancy include certain infections, such as rubella, toxins including valproic acid, alcohol, cocaine, pesticides, lead, and air pollution, fetal growth restriction, and autoimmune diseases. Controversies surround other proposed environmental causes; for example, the vaccine hypothesis, which has been disproven. Autism affects information processing in the brain and how nerve cells and their synapses connect and organize; how this occurs is not well understood. The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) combines forms of the condition, including Asperger syndrome and pervasive developmental disorder not otherwise specified (PDD-NOS) into the diagnosis of autism spectrum disorder (ASD).\n\nSeveral interventions have been shown to reduce symptoms and improve the ability of autistic people to function and participate independently in the community. Behavioral, psychological, education, and/or skill-building interventions may be used to assist autistic people to learn life skills necessary for living independently, as well as other social, communication, and language skills. Therapy also aims to reduce challenging behaviors and build upon strengths. Some autistic adults are unable to live independently. An autistic culture has developed, with some individuals seeking a cure and others believing autism should be accepted as a difference to be accommodated instead of cured.\n\nGlobally, autism is estimated to affect 24.8 million people . In the 2000s, the number of autistic people worldwide was estimated at 1–2 per 1,000 people. In the developed countries, about 1.5% of children are diagnosed with ASD , up from 0.7% in 2000 in the United States. It is diagnosed four to five times more often in males than females. The number of people diagnosed has increased considerably since the 1990s, which may be partly due to increased recognition of the condition.\n\nCharacteristics\nAutism is a highly variable neurodevelopmental disorder whose symptoms first appear during infancy or childhood, and generally follows a steady course without remission. Autistic people may be severely impaired in some respects but average, or even superior, in others. Overt symptoms gradually begin after the age of six months, become established by age two or three years and tend to continue through adulthood, although often in more muted form. It is distinguished by a characteristic triad of symptoms: impairments in social interaction, impairments in communication, and repetitive behavior. Other aspects, such as atypical eating, are also common but are not essential for diagnosis. Individual symptoms of autism occur in the general population and appear not to associate highly, without a sharp line separating pathologically severe from common traits.\n\nSocial development\nSocial deficits distinguish autism and the related autism spectrum disorders (ASD; see Classification) from other developmental disorders. Autistic people have social impairments and often lack the intuition about others that many people take for granted. Noted autistic Temple Grandin described her inability to understand the social communication of neurotypicals, or people with typical neural development, as leaving her feeling "like an anthropologist on Mars".\n\nUnusual social development becomes apparent early in childhood. Autistic infants show less attention to social stimuli, smile and look at others less often, and respond less to their own name. Autistic toddlers differ more strikingly from social norms; for example, they have less eye contact and turn-taking, and do not have the ability to use simple movements to express themselves, such as pointing at things. Three- to five-year-old autistic children are less likely to exhibit social understanding, approach others spontaneously, imitate and respond to emotions, communicate nonverbally, and take turns with others. However, they do form attachments to their primary caregivers. Most autistic children display moderately less attachment security than neurotypical children, although this difference disappears in children with higher mental development or less pronounced autistic traits. Older children and adults with ASD perform worse on tests of face and emotion recognition although this may be partly due to a lower ability to define a person\'s own emotions.\n\nChildren with high-functioning autism have more intense and frequent loneliness compared to non-autistic peers, despite the common belief that autistic children prefer to be alone. Making and maintaining friendships often proves to be difficult for autistic people. For them, the quality of friendships, not the number of friends, predicts how lonely they feel. Functional friendships, such as those resulting in invitations to parties, may affect the quality of life more deeply.\n\nThere are many anecdotal reports, but few systematic studies, of aggression and violence in individuals with ASD. The limited data suggest that, in children with intellectual disability, autism is associated with aggression, destruction of property, and meltdowns.\n\nCommunication\nAbout one third to half of autistic people do not develop enough natural speech to meet their daily communication needs. Differences in communication may be present from the first year of life, and may include delayed onset of babbling, unusual gestures, diminished responsiveness, and vocal patterns that are not synchronized with the caregiver. In the second and third years, autistic children have less frequent and less diverse babbling, consonants, words, and word combinations; their gestures are less often integrated with words. Autistic children are less likely to make requests or share experiences, and are more likely to simply repeat others\' words (echolalia) or reverse pronouns. Joint attention seems to be necessary for functional speech, and deficits in joint attention seem to distinguish infants with ASD. For example, they may look at a pointing hand instead of the object to which the hand is pointing, and they consistently fail to point at objects in order to comment on or share an experience. Autistic children may have difficulty with imaginative play and with developing symbols into language.\n\nIn a pair of studies, high-functioning autistic children aged 8–15 performed equally well as, and as adults better than, individually matched controls at basic language tasks involving vocabulary and spelling. Both autistic groups performed worse than controls at complex language tasks such as figurative language, comprehension, and inference. As people are often sized up initially from their basic language skills, these studies suggest that people speaking to autistic individuals are more likely to overestimate what their audience comprehends.\n\nRepetitive behavior\n\nAutistic individuals can display many forms of repetitive or restricted behavior, which the Repetitive Behavior Scale-Revised (RBS-R) categorizes as follows.\n\n Stereotyped behaviors: Repetitive movements, such as hand flapping, head rolling, or body rocking.\n Compulsive behaviors: Time-consuming behaviors intended to reduce the anxiety that an individual feels compelled to perform repeatedly or according to rigid rules, such as placing objects in a specific order, checking things, or handwashing.\n Sameness: Resistance to change; for example, insisting that the furniture not be moved or refusing to be interrupted.\n Ritualistic behavior: Unvarying pattern of daily activities, such as an unchanging menu or a dressing ritual. This is closely associated with sameness and an independent validation has suggested combining the two factors.\n Restricted interests: Interests or fixations that are abnormal in theme or intensity of focus, such as preoccupation with a single television program, toy, or game.\n Self-injury: Behaviors such as eye-poking, skin-picking, hand-biting and head-banging.\n\nNo single repetitive or self-injurious behavior seems to be specific to autism, but autism appears to have an elevated pattern of occurrence and severity of these behaviors.\n\nOther symptoms\nAutistic individuals may have symptoms that are independent of the diagnosis, but that can affect the individual or the family.\nAn estimated 0.5% to 10% of individuals with ASD show unusual abilities, ranging from splinter skills such as the memorization of trivia to the extraordinarily rare talents of prodigious autistic savants. Many individuals with ASD show superior skills in perception and attention, relative to the general population. Sensory abnormalities are found in over 90% of autistic people, and are considered core features by some, although there is no good evidence that sensory symptoms differentiate autism from other developmental disorders. Differences are greater for under-responsivity (for example, walking into things) than for over-responsivity (for example, distress from loud noises) or for sensation seeking (for example, rhythmic movements). An estimated 60–80% of autistic people have motor signs that include poor muscle tone, poor motor planning, and toe walking;  deficits in motor coordination are pervasive across ASD and are greater in autism proper. Unusual eating behavior occurs in about three-quarters of children with ASD, to the extent that it was formerly a diagnostic indicator. Selectivity is the most common problem, although eating rituals and food refusal also occur.\n\nThere is tentative evidence that gender dysphoria occurs more frequently in autistic people (see Autism and LGBT identities). As well as that, a 2021 anonymized online survey of 16-90 year-olds revealed that autistic males are more likely to be bisexual, while autistic females are more likely to be homosexual.\n\nGastrointestinal problems are one of the most commonly co-occurring medical conditions in autistic people. These are linked to greater social impairment, irritability, behavior and sleep problems, language impairments and mood changes.\n\nParents of children with ASD have higher levels of stress. Siblings of children with ASD report greater admiration of and less conflict with the affected sibling than siblings of unaffected children and were similar to siblings of children with Down syndrome in these aspects of the sibling relationship. However, they reported lower levels of closeness and intimacy than siblings of children with Down syndrome; siblings of individuals with ASD have greater risk of negative well-being and poorer sibling relationships as adults.\n\nCauses\n\nIt has long been presumed that there is a common cause at the genetic, cognitive, and neural levels for autism\'s characteristic triad of symptoms. However, there is increasing suspicion that autism is instead a complex disorder whose core aspects have distinct causes that often co-occur.\n\nAutism has a strong genetic basis, although the genetics of autism are complex and it is unclear whether ASD is explained more by rare mutations with major effects, or by rare multigene interactions of common genetic variants. Complexity arises due to interactions among multiple genes, the environment, and epigenetic factors which do not change DNA sequencing but are heritable and influence gene expression. Many genes have been associated with autism through sequencing the genomes of affected individuals and their parents. Studies of twins suggest that heritability is 0.7 for autism and as high as 0.9 for ASD, and siblings of those with autism are about 25 times more likely to be autistic than the general population. However, most of the mutations that increase autism risk have not been identified. Typically, autism cannot be traced to a Mendelian (single-gene) mutation or to a single chromosome abnormality, and none of the genetic syndromes associated with ASDs have been shown to selectively cause ASD. Numerous candidate genes have been located, with only small effects attributable to any particular gene. Most loci individually explain less than 1% of cases of autism. The large number of autistic individuals with unaffected family members may result from spontaneous structural variation—such as deletions, duplications or inversions in genetic material during meiosis. Hence, a substantial fraction of autism cases may be traceable to genetic causes that are highly heritable but not inherited: that is, the mutation that causes the autism is not present in the parental genome. Autism may be underdiagnosed in women and girls due to an assumption that it is primarily a male condition, but genetic phenomena such as imprinting and X linkage have the ability to raise the frequency and severity of conditions in males, and theories have been put forward for a genetic reason why males are diagnosed more often, such as the imprinted brain hypothesis and the extreme male brain theory.\n\nMaternal nutrition and inflammation during preconception and pregnancy influences fetal neurodevelopment. Intrauterine growth restriction is associated with ASD, in both term and preterm infants. Maternal inflammatory and autoimmune diseases may damage fetal tissues, aggravating a genetic problem or damaging the nervous system.\n\nExposure to air pollution during pregnancy, especially heavy metals and particulates, may increase the risk of autism. Environmental factors that have been claimed without evidence to contribute to or exacerbate autism include certain foods, infectious diseases, solvents, PCBs, phthalates and phenols used in plastic products, pesticides, brominated flame retardants, alcohol, smoking, illicit drugs, vaccines, and prenatal stress. Some, such as the MMR vaccine, have been completely disproven.\n\nParents may first become aware of autistic symptoms in their child around the time of a routine vaccination. This has led to unsupported theories blaming vaccine "overload", a vaccine preservative, or the MMR vaccine for causing autism. The latter theory was supported by a litigation-funded study that has since been shown to have been "an elaborate fraud". Although these theories lack convincing scientific evidence and are biologically implausible, parental concern about a potential vaccine link with autism has led to lower rates of childhood immunizations, outbreaks of previously controlled childhood diseases in some countries, and the preventable deaths of several children.\n\nMechanism\n\nAutism\'s symptoms result from maturation-related changes in various systems of the brain. How autism occurs is not well understood. Its mechanism can be divided into two areas: the pathophysiology of brain structures and processes associated with autism, and the neuropsychological linkages between brain structures and behaviors. The behaviors appear to have multiple pathophysiologies.\n\nThere is evidence that gut–brain axis abnormalities may be involved. A 2015 review proposed that immune dysregulation, gastrointestinal inflammation, malfunction of the autonomic nervous system, gut flora alterations, and food metabolites may cause brain neuroinflammation and dysfunction. A 2016 review concludes that enteric nervous system abnormalities might play a role in neurological disorders such as autism. Neural connections and the immune system are a pathway that may allow diseases originated in the intestine to spread to the brain.\n\nSeveral lines of evidence point to synaptic dysfunction as a cause of autism. Some rare mutations may lead to autism by disrupting some synaptic pathways, such as those involved with cell adhesion. Gene replacement studies in mice suggest that autistic symptoms are closely related to later developmental steps that depend on activity in synapses and on activity-dependent changes. All known teratogens (agents that cause birth defects) related to the risk of autism appear to act during the first eight weeks from conception, and though this does not exclude the possibility that autism can be initiated or affected later, there is strong evidence that autism arises very early in development.\n\nDiagnosis\nDiagnosis is based on behavior, not cause or mechanism. Under the DSM-5, autism is characterized by persistent deficits in social communication and interaction across multiple contexts, as well as restricted, repetitive patterns of behavior, interests, or activities. These deficits are present in early childhood, typically before age three, and lead to clinically significant functional impairment. Sample symptoms include lack of social or emotional reciprocity, stereotyped and repetitive use of language or idiosyncratic language, and persistent preoccupation with unusual objects. The disturbance must not be better accounted for by Rett syndrome, intellectual disability or global developmental delay. ICD-10 uses essentially the same definition.\n\nSeveral diagnostic instruments are available. Two are commonly used in autism research: the Autism Diagnostic Interview-Revised (ADI-R) is a semistructured parent interview, and the Autism Diagnostic Observation Schedule (ADOS) uses observation and interaction with the child. The Childhood Autism Rating Scale (CARS) is used widely in clinical environments to assess severity of autism based on observation of children. The Diagnostic interview for social and communication disorders (DISCO) may also be used.\n\nA pediatrician commonly performs a preliminary investigation by taking developmental history and physically examining the child. If warranted, diagnosis and evaluations are conducted with help from ASD specialists, observing and assessing cognitive, communication, family, and other factors using standardized tools, and taking into account any associated medical conditions. A pediatric neuropsychologist is often asked to assess behavior and cognitive skills, both to aid diagnosis and to help recommend educational interventions. A differential diagnosis for ASD at this stage might also consider intellectual disability, hearing impairment, and a specific language impairment such as Landau–Kleffner syndrome. The presence of autism can make it harder to diagnose coexisting psychiatric disorders such as depression.\n\nClinical genetics evaluations are often done once ASD is diagnosed, particularly when other symptoms already suggest a genetic cause. Although genetic technology allows clinical geneticists to link an estimated 40% of cases to genetic causes, consensus guidelines in the US and UK are limited to high-resolution chromosome and fragile X testing. A genotype-first model of diagnosis has been proposed, which would routinely assess the genome\'s copy number variations. As new genetic tests are developed several ethical, legal, and social issues will emerge. Commercial availability of tests may precede adequate understanding of how to use test results, given the complexity of autism\'s genetics. Metabolic and neuroimaging tests are sometimes helpful, but are not routine.\n\nASD can sometimes be diagnosed by age 14 months, although diagnosis becomes increasingly stable over the first three years of life: for example, a one-year-old who meets diagnostic criteria for ASD is less likely than a three-year-old to continue to do so a few years later. In the UK the National Autism Plan for Children recommends at most 30 weeks from first concern to completed diagnosis and assessment, though few cases are handled that quickly in practice. Although the symptoms of autism and ASD begin early in childhood, they are sometimes missed; years later, adults may seek diagnoses to help them or their friends and family understand themselves, to help their employers make adjustments, or in some locations to claim disability living allowances or other benefits.\n\nSigns of autism may be more challenging for clinicians to detect in females. Autistic females have been shown to engage in masking more frequently than autistic males. Masking may include making oneself perform normative facial expressions and eye contact. A notable percentage of autistic females may be misdiagnosed, diagnosed after a considerable delay, or not diagnosed at all.\n\nConversely, the cost of screening and diagnosis and the challenge of obtaining payment can inhibit or delay diagnosis. It is particularly hard to diagnose autism among the visually impaired, partly because some of its diagnostic criteria depend on vision, and partly because autistic symptoms overlap with those of common blindness syndromes or blindisms.\n\nClassification\nAutism is one of the five pervasive developmental disorders (PDD), which are characterized by widespread abnormalities of social interactions and communication, severely restricted interests, and highly repetitive behavior. These symptoms do not imply sickness, fragility, or emotional disturbance.\n\nOf the five PDD forms, Asperger syndrome is closest to autism in signs and likely causes; Rett syndrome and childhood disintegrative disorder share several signs with autism, but may have unrelated causes; PDD not otherwise specified (PDD-NOS; also called atypical autism) is diagnosed when the criteria are not met for a more specific disorder. Unlike with autism, people with Asperger syndrome have no substantial delay in language development. The terminology of autism can be bewildering, with autism, Asperger syndrome and PDD-NOS often called the autism spectrum disorders (ASD) or sometimes the autistic disorders, whereas autism itself is often called autistic disorder, childhood autism, or infantile autism. In this article, autism refers to the classic autistic disorder; in clinical practice, though, autism, ASD, and PDD are often used interchangeably. ASD, in turn, is a subset of the broader autism phenotype, which describes individuals who may not have ASD but do have autistic-like traits, such as avoiding eye contact.\n\nResearch into causes has been hampered by the inability to identify biologically meaningful subgroups within the autistic population and by the traditional boundaries between the disciplines of psychiatry, psychology, neurology and pediatrics. Newer technologies such as fMRI and diffusion tensor imaging can help identify biologically relevant phenotypes (observable traits) that can be viewed on brain scans, to help further neurogenetic studies of autism; one example is lowered activity in the fusiform face area of the brain, which is associated with impaired perception of people versus objects. It has been proposed to classify autism using genetics as well as behavior. (For more, see Brett Abrahams, geneticist and neuroscientist)\n\nSpectrum \nAutism has long been thought to cover a wide spectrum, ranging from individuals with severe impairments—who may be silent, developmentally disabled, and prone to frequent repetitive behavior such as hand flapping and rocking—to high functioning individuals who may have active but distinctly odd social approaches, narrowly focused interests, and verbose, pedantic communication. Because the behavior spectrum is continuous, boundaries between diagnostic categories are necessarily somewhat arbitrary.\n\nScreening\nAbout half of parents of children with ASD notice their child\'s unusual behaviors by age 18 months, and about four-fifths notice by age 24 months. According to an article, failure to meet any of the following milestones "is an absolute indication to proceed with further evaluations. Delay in referral for such testing may delay early diagnosis and treatment and affect the long-term outcome".\n No response to name (or eye-to-eye gaze) by 6 months. \n No babbling by 12 months.\n No gesturing (pointing, waving, etc.) by 12 months.\n No single words by 16 months.\n No two-word (spontaneous, not just echolalic) phrases by 24 months.\n Loss of any language or social skills, at any age.\n\nThe United States Preventive Services Task Force in 2016 found it was unclear if screening was beneficial or harmful among children in whom there is no concern. The Japanese practice is to screen all children for ASD at 18 and 24 months, using autism-specific formal screening tests. In contrast, in the UK, children whose families or doctors recognize possible signs of autism are screened. It is not known which approach is more effective. Screening tools include the Modified Checklist for Autism in Toddlers (M-CHAT), the Early Screening of Autistic Traits Questionnaire, and the First Year Inventory; initial data on M-CHAT and its predecessor, the Checklist for Autism in Toddlers (CHAT), on children aged 18–30 months suggests that it is best used in a clinical setting and that it has low sensitivity (many false-negatives) but good specificity (few false-positives). It may be more accurate to precede these tests with a broadband screener that does not distinguish ASD from other developmental disorders. Screening tools designed for one culture\'s norms for behaviors like eye contact may be inappropriate for a different culture. Although genetic screening for autism is generally still impractical, it can be considered in some cases, such as children with neurological symptoms and dysmorphic features.\n\nSome authors suggest that automatic motor assessment could be useful to screen the children with ASD for instance with behavioural motor and emotionals reactions during smartphone watching.\n\nPrevention\nWhile infection with rubella during pregnancy causes fewer than 1% of cases of autism, vaccination against rubella can prevent many of those cases.\n\nManagement\n\nThe main goals when treating autistic children are to lessen associated deficits and family distress, and to increase quality of life and functional independence. In general, higher IQs are correlated with greater responsiveness to treatment and improved treatment outcomes. No single treatment is best and treatment is typically tailored to the child\'s needs. Families and the educational system are the main resources for treatment. Services should be carried out by behavior analysts, special education teachers, speech pathologists, and licensed psychologists. Studies of interventions have methodological problems that prevent definitive conclusions about efficacy. However, the development of evidence-based interventions has advanced in recent years. Although many psychosocial interventions have some positive evidence, suggesting that some form of treatment is preferable to no treatment, the methodological quality of systematic reviews of these studies has generally been poor, their clinical results are mostly tentative, and there is little evidence for the relative effectiveness of treatment options. Intensive, sustained special education programs and behavior therapy early in life can help children acquire self-care, communication, and job skills, and often improve functioning and decrease symptom severity and maladaptive behaviors; claims that intervention by around age three years is crucial are not substantiated. While medications have not been found to help with core symptoms, they may be used for associated symptoms, such as irritability, inattention, or repetitive behavior patterns.\n\nEducation\nEducational interventions often used include applied behavior analysis (ABA), developmental models, structured teaching, speech and language therapy, social skills therapy, and occupational therapy and cognitive behavioral interventions in adults without intellectual disability to reduce depression, anxiety, and obsessive-compulsive disorder. Among these approaches, interventions either treat autistic features comprehensively, or focalize treatment on a specific area of deficit. The quality of research for early intensive behavioral intervention (EIBI)—a treatment procedure incorporating over thirty hours per week of the structured type of ABA that is carried out with very young children—is currently low, and more vigorous research designs with larger sample sizes are needed. Two theoretical frameworks outlined for early childhood intervention include structured and naturalistic ABA interventions, and developmental social pragmatic models (DSP). One interventional strategy utilizes a parent training model, which teaches parents how to implement various ABA and DSP techniques, allowing for parents to disseminate interventions themselves. Various DSP programs have been developed to explicitly deliver intervention systems through at-home parent implementation. Despite the recent development of parent training models, these interventions have demonstrated effectiveness in numerous studies, being evaluated as a probable efficacious mode of treatment.\n\nEarly, intensive ABA therapy has demonstrated effectiveness in enhancing communication and adaptive functioning in preschool children; it is also well-established for improving the intellectual performance of that age group. Similarly, a teacher-implemented intervention that utilizes a more naturalistic form of ABA combined with a developmental social pragmatic approach has been found to be beneficial in improving social-communication skills in young children, although there is less evidence in its treatment of global symptoms. Neuropsychological reports are often poorly communicated to educators, resulting in a gap between what a report recommends and what education is provided. It is not known whether treatment programs for children lead to significant improvements after the children grow up, and the limited research on the effectiveness of adult residential programs shows mixed results. The appropriateness of including children with varying severity of autism spectrum disorders in the general education population is a subject of current debate among educators and researchers.\n\nMedication\nMedications may be used to treat ASD symptoms that interfere with integrating a child into home or school when behavioral treatment fails. They may also be used for associated health problems, such as ADHD or anxiety. More than half of US children diagnosed with ASD are prescribed psychoactive drugs or anticonvulsants, with the most common drug classes being antidepressants, stimulants, and antipsychotics. The atypical antipsychotic drugs risperidone and aripiprazole are FDA-approved for treating associated aggressive and self-injurious behaviors. However, their side effects must be weighed against their potential benefits, and autistic people may respond atypically. Side effects, for example, may include weight gain, tiredness, drooling, and aggression. SSRI antidepressants, such as fluoxetine and fluvoxamine, have been shown to be effective in reducing repetitive and ritualistic behaviors, while the stimulant medication methylphenidate is beneficial for some children with co-morbid inattentiveness or hyperactivity. There is scant reliable research about the effectiveness or safety of drug treatments for adolescents and adults with ASD. No known medication relieves autism\'s core symptoms of social and communication impairments. Experiments in mice have reversed or reduced some symptoms related to autism by replacing or modulating gene function, suggesting the possibility of targeting therapies to specific rare mutations known to cause autism.\n\nAlternative medicine\nAlthough many alternative therapies and interventions are available, few are supported by scientific studies. Treatment approaches have little empirical support in quality-of-life contexts, and many programs focus on success measures that lack predictive validity and real-world relevance. Some alternative treatments may place the child at risk. The preference that autistic children have for unconventional foods can lead to reduction in bone cortical thickness with this being greater in those on casein-free diets, as a consequence of the low intake of calcium and vitamin D; however, suboptimal bone development in ASD has also been associated with lack of exercise and gastrointestinal disorders. In 2005, botched chelation therapy killed a five-year-old child with autism. Chelation is not recommended for autistic people since the associated risks outweigh any potential benefits. Another alternative medicine practice with no evidence is CEASE therapy, a mixture of homeopathy, supplements, and \'vaccine detoxing\'.\n\nAlthough popularly used as an alternative treatment for autistic people, as of 2018 there is no good evidence to recommend a gluten- and casein-free diet as a standard treatment. A 2018 review concluded that it may be a therapeutic option for specific groups of children with autism, such as those with known food intolerances or allergies, or with food intolerance markers. The authors analyzed the prospective trials conducted to date that studied the efficacy of the gluten- and casein-free diet in children with ASD (4 in total). All of them compared gluten- and casein-free diet versus normal diet with a control group (2 double-blind randomized controlled trials, 1 double-blind crossover trial, 1 single-blind trial). In two of the studies, whose duration was 12 and 24 months, a significant improvement in ASD symptoms (efficacy rate 50%) was identified. In the other two studies, whose duration was 3 months, no significant effect was observed. The authors concluded that a longer duration of the diet may be necessary to achieve the improvement of the ASD symptoms. Other problems documented in the trials carried out include transgressions of the diet, small sample size, the heterogeneity of the participants and the possibility of a placebo effect. In the subset of people who have gluten sensitivity there is limited evidence that suggests that a gluten-free diet may improve some autistic behaviors.\n\nResults of a systematic review on interventions to address health outcomes among autistic adults found emerging evidence to support mindfulness-based interventions for improving mental health. This includes decreasing stress, anxiety, ruminating thoughts, anger, and aggression. There is tentative evidence that music therapy may improve social interactions, verbal communication, and non-verbal communication skills. There has been early research looking at hyperbaric treatments in children with autism. Studies on pet therapy have shown positive effects.\n\nPrognosis\nThere is no known cure for autism. The degree of symptoms can decrease, occasionally to the extent that people lose their diagnosis of ASD; this occurs sometimes after intensive treatment and sometimes not. It is not known how often this outcome happens; reported rates in unselected samples have ranged from 3% to 25%. Most autistic children acquire language by age five or younger, though a few have developed communication skills in later years. Many autistic children lack social support, future employment opportunities or self-determination. Although core difficulties tend to persist, symptoms often become less severe with age.\n\nFew high-quality studies address long-term prognosis. Some adults show modest improvement in communication skills, but a few decline; no study has focused on autism after midlife. Acquiring language before age six, having an IQ above 50, and having a marketable skill all predict better outcomes; independent living is unlikely with severe autism.\n\nMany autistic people face significant obstacles in transitioning to adulthood. Compared to the general population autistic people are more likely to be unemployed and to have never had a job. About half of people in their 20s with autism are not employed.\n\nAutistic people tend to face increased stress levels related to psychosocial factors, such as stigma, which may increase the rates of mental health issues in the autistic population.\n\nEpidemiology\n\nAs of 2007, reviews estimate a prevalence of 1–2 per 1,000 for autism and close to 6 per 1,000 for ASD. A 2016 survey in the United States reported a rate of 25 per 1,000 children for ASD. Globally, autism affects an estimated 24.8 million people , while Asperger syndrome affects a further 37.2 million. In 2012, the NHS estimated that the overall prevalence of autism among adults aged 18 years and over in the UK was 1.1%. Rates of PDD-NOS\'s has been estimated at 3.7 per 1,000, Asperger syndrome at roughly 0.6 per 1,000, and childhood disintegrative disorder at 0.02 per 1,000. CDC estimates about 1 out of 59 (1.7%) for 2014, an increase from 1 out of every 68 children (1.5%) for 2010.\n\nIn the UK, from 1998 to 2018, the autism diagnoses increased by 787%. This increase is largely attributable to changes in diagnostic practices, referral patterns, availability of services, age at diagnosis, and public awareness (particularly among women), though unidentified environmental risk factors cannot be ruled out. The available evidence does not rule out the possibility that autism\'s true prevalence has increased; a real increase would suggest directing more attention and funding toward psychosocial factors and changing environmental factors instead of continuing to focus on genetics. It has been established that vaccination is not a risk factor for autism and is not behind any increase in autism prevalence rates, if any change in the rate of autism exists at all.\n\nMales are at higher risk for ASD than females. The sex ratio averages 4.3:1 and is greatly modified by cognitive impairment: it may be close to 2:1 with intellectual disability and more than 5.5:1 without. Several theories about the higher prevalence in males have been investigated, but the cause of the difference is unconfirmed; one theory is that females are underdiagnosed.\n\nAlthough the evidence does not implicate any single pregnancy-related risk factor as a cause of autism, the risk of autism is associated with advanced age in either parent, and with diabetes, bleeding, and use of psychiatric drugs in the mother during pregnancy. The risk is greater with older fathers than with older mothers; two potential explanations are the known increase in mutation burden in older sperm, and the hypothesis that men marry later if they carry genetic liability and show some signs of autism. Most professionals believe that race, ethnicity, and socioeconomic background do not affect the occurrence of autism.\n\nSeveral other conditions are common in children with autism. They include:\n Genetic disorders. About 10–15% of autism cases have an identifiable Mendelian (single-gene) condition, chromosome abnormality, or other genetic syndrome, and ASD is associated with several genetic disorders.\n Intellectual disability. The percentage of autistic individuals who also meet criteria for intellectual disability has been reported as anywhere from 25% to 70%, a wide variation illustrating the difficulty of assessing intelligence of individuals on the autism spectrum. In comparison, for PDD-NOS the association with intellectual disability is much weaker, and by definition, the diagnosis of Asperger\'s excludes intellectual disability.\n Anxiety disorders are common among children with ASD; there are no firm data, but studies have reported prevalences ranging from 11% to 84%. Many anxiety disorders have symptoms that are better explained by ASD itself, or are hard to distinguish from ASD\'s symptoms.\n Epilepsy, with variations in risk of epilepsy due to age, cognitive level, and type of language disorder.\n Several metabolic defects, such as phenylketonuria, are associated with autistic symptoms.\n Minor physical anomalies are significantly increased in the autistic population.\n Preempted diagnoses. Although the DSM-IV rules out the concurrent diagnosis of many other conditions along with autism, the full criteria for Attention deficit hyperactivity disorder (ADHD), Tourette syndrome, and other of these conditions are often present and these co-occurrent conditions are increasingly accepted.\n Sleep problems affect about two-thirds of individuals with ASD at some point in childhood. These most commonly include symptoms of insomnia such as difficulty in falling asleep, frequent nocturnal awakenings, and early morning awakenings. Sleep problems are associated with difficult behaviors and family stress, and are often a focus of clinical attention over and above the primary ASD diagnosis.\n\nHistory\n\nA few examples of autistic symptoms and treatments were described long before autism was named. The Table Talk of Martin Luther, compiled by his notetaker, Mathesius, contains the story of a 12-year-old boy who may have been severely autistic. The earliest well-documented case of autism is that of Hugh Blair of Borgue, as detailed in a 1747 court case in which his brother successfully petitioned to annul Blair\'s marriage to gain Blair\'s inheritance. The Wild Boy of Aveyron, a feral child caught in 1798, showed several signs of autism; the medical student Jean Itard treated him with a behavioral program designed to help him form social attachments and to induce speech via imitation.\n\nThe New Latin word autismus (English translation autism) was coined by the Swiss psychiatrist Eugen Bleuler in 1910 as he was defining symptoms of schizophrenia. He derived it from the Greek word autós (αὐτός, meaning "self"), and used it to mean morbid self-admiration, referring to "autistic withdrawal of the patient to his fantasies, against which any influence from outside becomes an intolerable disturbance". A Soviet child psychiatrist, Grunya Sukhareva, described a similar syndrome that was published in Russian in 1925, and in German in 1926.\n\nClinical development and diagnoses \n\nThe word autism first took its modern sense in 1938 when Hans Asperger of the Vienna University Hospital adopted Bleuler\'s terminology autistic psychopaths in a lecture in German about child psychology. Asperger was investigating an ASD now known as Asperger syndrome, though for various reasons it was not widely recognized as a separate diagnosis until 1981. Leo Kanner of the Johns Hopkins Hospital first used autism in its modern sense in English when he introduced the label early infantile autism in a 1943 report of 11 children with striking behavioral similarities. Almost all the characteristics described in Kanner\'s first paper on the subject, notably "autistic aloneness" and "insistence on sameness", are still regarded as typical of the autistic spectrum of disorders. It is not known whether Kanner derived the term independently of Asperger.\n\nKanner\'s reuse of autism led to decades of confused terminology like infantile schizophrenia, and child psychiatry\'s focus on maternal deprivation led to misconceptions of autism as an infant\'s response to "refrigerator mothers". Starting in the late 1960s autism was established as a separate syndrome.\n\nTerminology and distinction from schizophrenia \nAs late as the mid-1970s there was little evidence of a genetic role in autism, while in 2007 it was believed to be one of the most heritable psychiatric conditions. Although the rise of parent organizations and the destigmatization of childhood ASD have affected how ASD is viewed, parents continue to feel social stigma in situations where their child\'s autistic behavior is perceived negatively, and many primary care physicians and medical specialists express some beliefs consistent with outdated autism research.\n\nIt took until 1980 for the DSM-III to differentiate autism from childhood schizophrenia. In 1987, the DSM-III-R provided a checklist for diagnosing autism. In May 2013, the DSM-5 was released, updating the classification for pervasive developmental disorders. The grouping of disorders, including PDD-NOS, autism, Asperger syndrome, Rett syndrome, and CDD, has been removed and replaced with the general term of Autism Spectrum Disorders. The two categories that exist are impaired social communication and/or interaction, and restricted and/or repetitive behaviors.\n\nThe Internet has helped autistic individuals bypass nonverbal cues and emotional sharing that they find difficult to deal with, and has given them a way to form online communities and work remotely. Societal and cultural aspects of autism have developed: some in the community seek a cure, while others believe that autism is simply another way of being.\n\nSociety and culture\n\nAn autistic culture has emerged, accompanied by the autistic rights and neurodiversity movements. Events include World Autism Awareness Day, Autism Sunday, Autistic Pride Day, Autreat, and others. Social-science scholars study those with autism in hopes to learn more about "autism as a culture, transcultural comparisons\xa0... and research on social movements." Many autistic individuals have been successful in their fields.\n\nAutism rights movement \nThe autism rights movement is a social movement within the context of disability rights that emphasizes the concept of neurodiversity, viewing the autism spectrum as a result of natural variations in the human brain rather than a disorder to be cured. The autism rights movement advocates for including greater acceptance of autistic behaviors; therapies that focus on coping skills rather than on imitating the behaviors of those without autism, and the recognition of the autistic community as a minority group. Autism rights or neurodiversity advocates believe that the autism spectrum is genetic and should be accepted as a natural expression of the human genome. This perspective is distinct from fringe theories that autism is caused by environmental factors such as vaccines. A common criticism against autistic activists is that the majority of them are "high-functioning" or have Asperger syndrome and do not represent the views of "low-functioning" autistic people.\n\nEmployment\nAbout half of autistic people are unemployed, and one third of those with graduate degrees may be unemployed. Among those who find work, most are employed in sheltered settings working for wages below the national minimum. While employers state hiring concerns about productivity and supervision, experienced employers of autistic people give positive reports of above average memory and detail orientation as well as a high regard for rules and procedure in autistic employees. A majority of the economic burden of autism is caused by decreased earnings in the job market. Some studies also find decreased earning among parents who care for autistic children.\n\nReferences\n\nExternal links\n\n \n1910s neologisms\nArticles containing video clips\nCommunication disorders\nNeurological disorders in children\nPervasive developmental disorders\nWikipedia medicine articles ready to translate', 'Albedo (; ) is the measure of the diffuse reflection of solar radiation out of the total solar radiation and measured on a scale from 0, corresponding to a black body that absorbs all incident radiation, to 1, corresponding to a body that reflects all incident radiation.\n\nSurface albedo is defined as the ratio of radiosity Je to the irradiance Ee (flux per unit area) received by a surface. The proportion reflected is not only determined by properties of the surface itself, but also by the spectral and angular distribution of solar radiation reaching the Earth\'s surface. These factors vary with atmospheric composition, geographic location, and time (see position of the Sun). While bi-hemispherical reflectance is calculated for a single angle of incidence (i.e., for a given position of the Sun), albedo is the directional integration of reflectance over all solar angles in a given period. The temporal resolution may range from seconds (as obtained from flux measurements) to daily, monthly, or annual averages.\n\nUnless given for a specific wavelength (spectral albedo), albedo refers to the entire spectrum of solar radiation. Due to measurement constraints, it is often given for the spectrum in which most solar energy reaches the surface (between 0.3 and 3 μm). This spectrum includes visible light (0.4–0.7 μm), which explains why surfaces with a low albedo appear dark (e.g., trees absorb most radiation), whereas surfaces with a high albedo appear bright (e.g., snow reflects most radiation).\n\nAlbedo is an important concept in climatology, astronomy, and environmental management (e.g., as part of the Leadership in Energy and Environmental Design (LEED) program for sustainable rating of buildings). The average albedo of the Earth from the upper atmosphere, its planetary albedo, is 30–35% because of cloud cover, but widely varies locally across the surface because of different geological and environmental features.\n\nThe term albedo was introduced into optics by Johann Heinrich Lambert in his 1760 work Photometria.\n\nTerrestrial albedo\n\nAny albedo in visible light falls within a range of about 0.9 for fresh snow to about 0.04 for charcoal, one of the darkest substances. Deeply shadowed cavities can achieve an effective albedo approaching the zero of a black body. When seen from a distance, the ocean surface has a low albedo, as do most forests, whereas desert areas have some of the highest albedos among landforms. Most land areas are in an albedo range of 0.1 to 0.4. The average albedo of Earth is about 0.3. This is far higher than for the ocean primarily because of the contribution of clouds.\n\nEarth\'s surface albedo is regularly estimated via Earth observation satellite sensors such as NASA\'s MODIS instruments on board the Terra and Aqua satellites, and the CERES instrument on the Suomi NPP and JPSS. As the amount of reflected radiation is only measured for a single direction by satellite, not all directions, a mathematical model is used to translate a sample set of satellite reflectance measurements into estimates of directional-hemispherical reflectance and bi-hemispherical reflectance (e.g.,). These calculations are based on the bidirectional reflectance distribution function (BRDF), which describes how the reflectance of a given surface depends on the view angle of the observer and the solar angle. BDRF can facilitate translations of observations of reflectance into albedo.\n\nEarth\'s average surface temperature due to its albedo and the greenhouse effect is currently about . If Earth were frozen entirely (and hence be more reflective), the average temperature of the planet would drop below . If only the continental land masses became covered by glaciers, the mean temperature of the planet would drop to about . In contrast, if the entire Earth was covered by water – a so-called ocean planet – the average temperature on the planet would rise to almost .\n\nIn 2021, scientists reported that Earth dimmed by ~0.5% over two decades (1998-2017) as measured by earthshine using modern photometric techniques. This may have both been co-caused by climate change as well as a substantial increase in global warming. However, the link to climate change has not been explored to date and it is unclear whether or not this represents an ongoing trend.\n\nWhite-sky, black-sky, and blue-sky albedo\nFor land surfaces, it has been shown that the albedo at a particular solar zenith angle θi can be approximated by the proportionate sum of two terms:\n the directional-hemispherical reflectance at that solar zenith angle, , sometimes referred to as black-sky albedo, and\n the bi-hemispherical reflectance, , sometimes referred to as white-sky albedo.\nwith  being the proportion of direct radiation from a given solar angle, and  being the proportion of diffuse illumination, the actual albedo  (also called blue-sky albedo) can then be given as:\n\nThis formula is important because it allows the albedo to be calculated for any given illumination conditions from a knowledge of the intrinsic properties of the surface.\n\nExamples of terrestrial albedo effects\n\nIllumination\nAlbedo is not directly dependent on illumination because changing the amount of incoming light proportionally changes the amount of reflected light, except in circumstances where a change in illumination induces a change in the Earth\'s surface at that location (e.g. through melting of reflective ice). That said, albedo and illumination both vary by latitude. Albedo is highest near the poles and lowest in the subtropics, with a local maximum in the tropics.\n\nInsolation effects\nThe intensity of albedo temperature effects depends on the amount of albedo and the level of local insolation (solar irradiance); high albedo areas in the Arctic and Antarctic regions are cold due to low insolation, whereas areas such as the Sahara Desert, which also have a relatively high albedo, will be hotter due to high insolation. Tropical and sub-tropical rainforest areas have low albedo, and are much hotter than their temperate forest counterparts, which have lower insolation. Because insolation plays such a big role in the heating and cooling effects of albedo, high insolation areas like the tropics will tend to show a more pronounced fluctuation in local temperature when local albedo changes.\n\nArctic regions notably release more heat back into space than what they absorb, effectively cooling the Earth. This has been a concern since arctic ice and snow has been melting at higher rates due to higher temperatures, creating regions in the arctic that are notably darker (being water or ground which is darker color) and reflects less heat back into space. This feedback loop results in a reduced albedo effect.\n\nClimate and weather\nAlbedo affects climate by determining how much radiation a planet absorbs. The uneven heating of Earth from albedo variations between land, ice, or ocean surfaces can drive weather.\n\nAlbedo–temperature feedback\nWhen an area\'s albedo changes due to snowfall, a snow–temperature feedback results. A layer of snowfall increases local albedo, reflecting away sunlight, leading to local cooling. In principle, if no outside temperature change affects this area (e.g., a warm air mass), the raised albedo and lower temperature would maintain the current snow and invite further snowfall, deepening the snow–temperature feedback. However, because local weather is dynamic due to the change of seasons, eventually warm air masses and a more direct angle of sunlight (higher insolation) cause melting. When the melted area reveals surfaces with lower albedo, such as grass, soil, or ocean, the effect is reversed: the darkening surface lowers albedo, increasing local temperatures, which induces more melting and thus reducing the albedo further, resulting in still more heating.\n\nSnow\nSnow albedo is highly variable, ranging from as high as 0.9 for freshly fallen snow, to about 0.4 for melting snow, and as low as 0.2 for dirty snow. Over Antarctica snow albedo averages a little more than 0.8. If a marginally snow-covered area warms, snow tends to melt, lowering the albedo, and hence leading to more snowmelt because more radiation is being absorbed by the snowpack (the ice–albedo positive feedback).\n\nJust as fresh snow has a higher albedo than does dirty snow, the albedo of snow-covered sea ice is far higher than that of sea water. Sea water absorbs more solar radiation than would the same surface covered with reflective snow. When sea ice melts, either due to a rise in sea temperature or in response to increased solar radiation from above, the snow-covered surface is reduced, and more surface of sea water is exposed, so the rate of energy absorption increases. The extra absorbed energy heats the sea water, which in turn increases the rate at which sea ice melts. As with the preceding example of snowmelt, the process of melting of sea ice is thus another example of a positive feedback. Both positive feedback loops have long been recognized as important for global warming.\n\nCryoconite, powdery windblown dust containing soot, sometimes reduces albedo on glaciers and ice sheets.\n\nThe dynamical nature of albedo in response to positive feedback, together with the effects of small errors in the measurement of albedo, can lead to large errors in energy estimates. Because of this, in order to reduce the error of energy estimates, it is important to measure the albedo of snow-covered areas through remote sensing techniques rather than applying a single value for albedo over broad regions.\n\nSmall-scale effects\nAlbedo works on a smaller scale, too. In sunlight, dark clothes absorb more heat and light-coloured clothes reflect it better, thus allowing some control over body temperature by exploiting the albedo effect of the colour of external clothing.\n\nSolar photovoltaic effects \nAlbedo can affect the electrical energy output of solar photovoltaic devices. For example, the effects of a spectrally responsive albedo are illustrated by the differences between the spectrally weighted albedo of solar photovoltaic technology based on hydrogenated amorphous silicon (a-Si:H) and crystalline silicon (c-Si)-based compared to traditional spectral-integrated albedo predictions. Research showed impacts of over 10%. More recently, the analysis was extended to the effects of spectral bias due to the specular reflectivity of 22 commonly occurring surface materials (both human-made and natural) and analyzes the albedo effects on the performance of seven photovoltaic materials covering three common photovoltaic system topologies: industrial (solar farms), commercial flat rooftops and residential pitched-roof applications.\n\nTrees\nBecause forests generally have a low albedo, (the majority of the ultraviolet and visible spectrum is absorbed through photosynthesis), some scientists have suggested that greater heat absorption by trees could offset some of the carbon benefits of afforestation (or offset the negative climate impacts of deforestation). In the case of evergreen forests with seasonal snow cover albedo reduction may be great enough for deforestation to cause a net cooling effect. Trees also impact climate in extremely complicated ways through evapotranspiration. The water vapor causes cooling on the land surface, causes heating where it condenses, acts a strong greenhouse gas, and can increase albedo when it condenses into clouds. Scientists generally treat evapotranspiration as a net cooling impact, and the net climate impact of albedo and evapotranspiration changes from deforestation depends greatly on local climate.\n\nIn seasonally snow-covered zones, winter albedos of treeless areas are 10% to 50% higher than nearby forested areas because snow does not cover the trees as readily. Deciduous trees have an albedo value of about 0.15 to 0.18 whereas coniferous trees have a value of about 0.09 to 0.15. Variation in summer albedo across both forest types is associated with maximum rates of photosynthesis because plants with high growth capacity display a greater fraction of their foliage for direct interception of incoming radiation in the upper canopy. The result is that wavelengths of light not used in photosynthesis are more likely to be reflected back to space rather than being absorbed by other surfaces lower in the canopy.\n\nStudies by the Hadley Centre have investigated the relative (generally warming) effect of albedo change and (cooling) effect of carbon sequestration on planting forests. They found that new forests in tropical and midlatitude areas tended to cool; new forests in high latitudes (e.g., Siberia) were neutral or perhaps warming.\n\nWater\n\nWater reflects light very differently from typical terrestrial materials. The reflectivity of a water surface is calculated using the Fresnel equations.\n\nAt the scale of the wavelength of light even wavy water is always smooth so the light is reflected in a locally specular manner (not diffusely). The glint of light off water is a commonplace effect of this. At small angles of incident light, waviness results in reduced reflectivity because of the steepness of the reflectivity-vs.-incident-angle curve and a locally increased average incident angle.\n\nAlthough the reflectivity of water is very low at low and medium angles of incident light, it becomes very high at high angles of incident light such as those that occur on the illuminated side of Earth near the terminator (early morning, late afternoon, and near the poles). However, as mentioned above, waviness causes an appreciable reduction. Because light specularly reflected from water does not usually reach the viewer, water is usually considered to have a very low albedo in spite of its high reflectivity at high angles of incident light.\n\nNote that white caps on waves look white (and have high albedo) because the water is foamed up, so there are many superimposed bubble surfaces which reflect, adding up their reflectivities. Fresh \'black\' ice exhibits Fresnel reflection.\nSnow on top of this sea ice increases the albedo to 0.9.\n\nClouds\nCloud albedo has substantial influence over atmospheric temperatures. Different types of clouds exhibit different reflectivity, theoretically ranging in albedo from a minimum of near 0 to a maximum approaching 0.8. "On any given day, about half of Earth is covered by clouds, which reflect more sunlight than land and water. Clouds keep Earth cool by reflecting sunlight, but they can also serve as blankets to trap warmth."\n\nAlbedo and climate in some areas are affected by artificial clouds, such as those created by the contrails of heavy commercial airliner traffic. A study following the burning of the Kuwaiti oil fields during Iraqi occupation showed that temperatures under the burning oil fires were as much as  colder than temperatures several miles away under clear skies.\n\nAerosol effects\nAerosols (very fine particles/droplets in the atmosphere) have both direct and indirect effects on Earth\'s radiative balance. The direct (albedo) effect is generally to cool the planet; the indirect effect (the particles act as cloud condensation nuclei and thereby change cloud properties) is less certain. As per Spracklen et al. the effects are:\n Aerosol direct effect. Aerosols directly scatter and absorb radiation. The scattering of radiation causes atmospheric cooling, whereas absorption can cause atmospheric warming.\n Aerosol indirect effect. Aerosols modify the properties of clouds through a subset of the aerosol population called cloud condensation nuclei. Increased nuclei concentrations lead to increased cloud droplet number concentrations, which in turn leads to increased cloud albedo, increased light scattering and radiative cooling (first indirect effect), but also leads to reduced precipitation efficiency and increased lifetime of the cloud (second indirect effect).\n\nIn extremely polluted cities like Delhi, aerosol pollutants influence local weather and induce an urban cool island effect during the day.\n\nBlack carbon\nAnother albedo-related effect on the climate is from black carbon particles. The size of this effect is difficult to quantify: the Intergovernmental Panel on Climate Change estimates that the global mean radiative forcing for black carbon aerosols from fossil fuels is +0.2 W m−2, with a range +0.1 to +0.4 W m−2. Black carbon is a bigger cause of the melting of the polar ice cap in the Arctic than carbon dioxide due to its effect on the albedo.\n\nHuman activities\nHuman activities (e.g., deforestation, farming, and urbanization) change the albedo of various areas around the globe. However, quantification of this effect on the global scale is difficult, further study is required to determine anthropogenic effects.\n\nAlbedo in Astronomy \nIn astronomy, the term albedo can be defined in several different ways, depending upon the application and the wavelength of electromagnetic radiation involved.\n\nOptical or Visual Albedo\nThe albedos of planets, satellites and minor planets such as asteroids can be used to infer much about their properties. The study of albedos, their dependence on wavelength, lighting angle ("phase angle"), and variation in time composes a major part of the astronomical field of photometry. For small and far objects that cannot be resolved by telescopes, much of what we know comes from the study of their albedos. For example, the absolute albedo can indicate the surface ice content of outer Solar System objects, the variation of albedo with phase angle gives information about regolith properties, whereas unusually high radar albedo is indicative of high metal content in asteroids.\n\nEnceladus, a moon of Saturn, has one of the highest known optical albedos of any body in the Solar System, with an albedo of 0.99. Another notable high-albedo body is Eris, with an albedo of 0.96. Many small objects in the outer Solar System and asteroid belt have low albedos down to about 0.05. A typical comet nucleus has an albedo of 0.04. Such a dark surface is thought to be indicative of a primitive and heavily space weathered surface containing some organic compounds.\n\nThe overall albedo of the Moon is measured to be around 0.14, but it is strongly directional and non-Lambertian, displaying also a strong opposition effect. Although such reflectance properties are different from those of any terrestrial terrains, they are typical of the regolith surfaces of airless Solar System bodies.\n\nTwo common optical albedos that are used in astronomy are the (V-band) geometric albedo (measuring brightness when illumination comes from directly behind the observer) and the Bond albedo (measuring total proportion of electromagnetic energy reflected). Their values can differ significantly, which is a common source of confusion.\n\nIn detailed studies, the directional reflectance properties of astronomical bodies are often expressed in terms of the five Hapke parameters which semi-empirically describe the variation of albedo with phase angle, including a characterization of the opposition effect of regolith surfaces. One of these five parameters is yet another type of albedo called the single-scattering albedo. It is used to define scattering of electromagnetic waves on small particles. It depends on properties of the material (refractive index), the size of the particle, and the wavelength of the incoming radiation. \n\nAn important relationship between an object\'s astronomical (geometric) albedo, absolute magnitude and diameter is given by:\n\nwhere  is the astronomical albedo,  is the diameter in kilometers, and  is the absolute magnitude.\n\nRadar Albedo\nIn planetary radar astronomy, a microwave (or radar) pulse is transmitted toward a planetary target (e.g. Moon, asteroid, etc.) and the echo from the target is measured. In most instances, the transmitted pulse is circularly polarized and the received pulse is measured in the same sense of polarization as the transmitted pulse (SC) and the opposite sense (OC). The echo power is measured in terms of radar cross-section, , , or  (total power, SC + OC) and is equal to the cross-sectional area of a metallic sphere (perfect reflector) at the same distance as the target that would return the same echo power.\n\nThose components of the received echo that return from first-surface reflections (as from a smooth or mirror-like surface) are dominated by the OC component as there is a reversal in polarization upon reflection. If the surface is rough at the wavelength scale or there is significant penetration into the regolith, there will be a significant SC component in the echo caused by multiple scattering.\n\nFor most objects in the solar system, the OC echo dominates and the most commonly reported radar albedo parameter is the (normalized) OC radar albedo (often shortened to radar albedo):\n\nwhere the denominator is the effective cross-sectional area of the target object with mean radius, . A smooth metallic sphere would have .\n\nRadar Albedos of Solar System Objects\n\nThe values reported for the Moon, Mercury, Mars, Venus, and Comet P/2005 JQ5 are derived from the total (OC+SC) radar albedo reported in those references.\n\nRelationship to Surface Bulk Density\nIn the event that most of the echo is from first surface reflections ( or so), the OC radar albedo is a first-order approximation of the Fresnel reflection coefficient (aka reflectivity) and can be used to estimate the bulk density of a planetary surface to a depth of a meter or so (a few wavelengths of the radar wavelength which is typically at the decimeter scale) using the following empirical relationships:\n\n \n\n.\n\nSee also\n\n Cool roof\n Daisyworld\n Emissivity\n Exitance\n Global dimming\n Irradiance\n Kirchhoff\'s law of thermal radiation\n Opposition surge\n Polar see-saw\n Radar astronomy\n Solar radiation management\n\nReferences\n\nExternal links\n\n Albedo Project\n Albedo – Encyclopedia of Earth\n NASA MODIS BRDF/albedo product site\n Ocean surface albedo look-up-table\n Surface albedo derived from Meteosat observations\n A discussion of Lunar albedos\n reflectivity of metals (chart)\n\nLand surface effects on climate\nClimate change feedbacks\nClimate forcing\nClimatology\nElectromagnetic radiation\nRadiometry\nScattering, absorption and radiative transfer (optics)\nRadiation\n1760s neologisms', 'A, or a, is the first letter and the first vowel of the modern English alphabet and the ISO basic Latin alphabet. Its name in English is a (pronounced ), plural aes. It is similar in shape to the Ancient Greek letter alpha, from which it derives. The uppercase version consists of the two slanting sides of a triangle, crossed in the middle by a horizontal bar. The lowercase version can be written in two forms: the double-storey a and single-storey ɑ. The latter is commonly used in handwriting and fonts based on it, especially fonts intended to be read by children, and is also found in italic type.\n\nIn the English grammar, "a", and its variant "an", are indefinite articles.\n\nHistory\n\nThe earliest certain ancestor of "A" is aleph (also written \'aleph), the first letter of the Phoenician alphabet, which consisted entirely of consonants (for that reason, it is also called an abjad to distinguish it from a true alphabet). In turn, the ancestor of aleph may have been a pictogram of an ox head in proto-Sinaitic script influenced by Egyptian hieroglyphs, styled as a triangular head with two horns extended.\n\nWhen the ancient Greeks adopted the alphabet, they had no use for a letter to represent the glottal stop—the consonant sound that the letter denoted in Phoenician and other Semitic languages, and that was the first phoneme of the Phoenician pronunciation of the letter—so they used their version of the sign to represent the vowel , and called it by the similar name of alpha. In the earliest Greek inscriptions after the Greek Dark Ages, dating to the 8th century BC, the letter rests upon its side, but in the Greek alphabet of later times it generally resembles the modern capital letter, although many local varieties can be distinguished by the shortening of one leg, or by the angle at which the cross line is set.\n\nThe Etruscans brought the Greek alphabet to their civilization in the Italian Peninsula and left the letter unchanged. The Romans later adopted the Etruscan alphabet to write the Latin language, and the resulting letter was preserved in the Latin alphabet that would come to be used to write many languages, including English.\n\nTypographic variants\n\nDuring Roman times, there were many variant forms of the letter "A". First was the monumental or lapidary style, which was used when inscribing on stone or other "permanent" media. There was also a cursive style used for everyday or utilitarian writing, which was done on more perishable surfaces. Due to the "perishable" nature of these surfaces, there are not as many examples of this style as there are of the monumental, but there are still many surviving examples of different types of cursive, such as majuscule cursive, minuscule cursive, and semicursive minuscule. Variants also existed that were intermediate between the monumental and cursive styles. The known variants include the early semi-uncial, the uncial, and the later semi-uncial.\n\nAt the end of the Roman Empire (5th century AD), several variants of the cursive minuscule developed through Western Europe. Among these were the semicursive minuscule of Italy, the Merovingian script in France, the Visigothic script in Spain, and the Insular or Anglo-Irish semi-uncial or Anglo-Saxon majuscule of Great Britain. By the 9th century, the Caroline script, which was very similar to the present-day form, was the principal form used in book-making, before the advent of the printing press. This form was derived through a combining of prior forms.\n\n15th-century Italy saw the formation of the two main variants that are known today. These variants, the Italic and Roman forms, were derived from the Caroline Script version. The Italic form, also called script a, is used in most current handwriting; it consists of a circle and vertical stroke on the right ("ɑ"). This slowly developed from the fifth-century form resembling the Greek letter tau in the hands of medieval Irish and English writers. The Roman form is used in most printed material; it consists of a small loop with an arc over it ("a"). Both derive from the majuscule (capital) form. In Greek handwriting, it was common to join the left leg and horizontal stroke into a single loop, as demonstrated by the uncial version shown. Many fonts then made the right leg vertical. In some of these, the serif that began the right leg stroke developed into an arc, resulting in the printed form, while in others it was dropped, resulting in the modern handwritten form. Graphic designers refer to the Italic and Roman forms as "single decker a" and "double decker a" respectively.\n\nItalic type is commonly used to mark emphasis or more generally to distinguish one part of a text from the rest (set in Roman type). There are some other cases aside from italic type where script a ("ɑ"), also called Latin alpha, is used in contrast with Latin "a" (such as in the International Phonetic Alphabet).\n\nUse in writing systems\n\nEnglish\n\nIn modern English orthography, the letter  represents at least seven different vowel sounds:\nthe near-open front unrounded vowel  as in pad;\nthe open back unrounded vowel  as in father, which is closer to its original Latin and Greek sound;\nthe diphthong  as in ace and major (usually when  is followed by one, or occasionally two, consonants and then another vowel letter) – this results from Middle English lengthening followed by the Great Vowel Shift;\nthe modified form of the above sound that occurs before , as in square and Mary;\nthe rounded vowel of water;\nthe shorter rounded vowel (not present in General American) in was and what;\na schwa, in many unstressed syllables, as in about, comma, solar.\n\nThe double  sequence does not occur in native English words, but is found in some words derived from foreign languages such as Aaron and aardvark. However,  occurs in many common digraphs, all with their own sound or sounds, particularly , , , ,  and .\n\n is the third-most-commonly used letter in English (after  and ) and French, the second most common in Spanish, and the most common in Portuguese. About 8.167% of letters used in English texts tend to be ; the number is around 7.636% in French, 11.525% in Spanish, and 14.634% for Portuguese.\n\nOther languages\nIn most languages that use the Latin alphabet,  denotes an open unrounded vowel, such as , , or . An exception is Saanich, in which  (and the glyph Á) stands for a close-mid front unrounded vowel .\n\nOther systems\n\nIn phonetic and phonemic notation:\nin the International Phonetic Alphabet,  is used for the open front unrounded vowel,  is used for the open central unrounded vowel, and  is used for the open back unrounded vowel.\nin X-SAMPA,  is used for the open front unrounded vowel and  is used for the open back unrounded vowel.\n\nOther uses\n\nIn algebra, the letter a along with various other letters of the alphabet is often used to denote a variable, with various conventional meanings in different areas of mathematics. Moreover, in 1637, René Descartes "invented the convention of representing unknowns in equations by x, y, and z, and knowns by a, b, and c", and this convention is still often followed, especially in elementary algebra.\n\nIn geometry, capital A, B, C etc. are used to denote segments, lines, rays, etc. A capital A is also typically used as one of the letters to represent an angle in a triangle, the lowercase a representing the side opposite angle A.\n\n"A" is often used to denote something or someone of a better or more prestigious quality or status: A-, A or A+, the best grade that can be assigned by teachers for students\' schoolwork; "A grade" for clean restaurants; A-list celebrities, etc. Such associations can have a motivating effect, as exposure to the letter A has been found to improve performance, when compared with other letters.\n\n"A" is used as a prefix on some words, such as asymmetry, to mean "not" or "without" (from Greek).\n\nIn English grammar, "a", and its variant "an", is an indefinite article, used to introduce noun phrases.\n\nFinally, the letter A is used to denote size, as in a narrow size shoe, or a small cup size in a brassiere.\n\nRelated characters\n\nDescendants and related characters in the Latin alphabet\nÆ æ : Latin AE ligature\nA with diacritics: Å å Ǻ ǻ Ḁ ḁ ẚ Ă ă Ặ ặ Ắ ắ Ằ ằ Ẳ ẳ Ẵ ẵ Ȃ ȃ Â â Ậ ậ Ấ ấ Ầ ầ Ẫ ẫ Ẩ ẩ Ả ả Ǎ ǎ Ⱥ ⱥ Ȧ ȧ Ǡ ǡ Ạ ạ Ä ä Ǟ ǟ À à Ȁ ȁ Á á Ā ā Ā̀ ā̀ Ã ã Ą ą Ą́ ą́ Ą̃ ą̃ A̲ a̲ ᶏ\nPhonetic alphabet symbols related to A (the International Phonetic Alphabet only uses lowercase, but uppercase forms are used in some other writing systems): \nⱭ ɑ : Latin letter alpha / script A, which represents an open back unrounded vowel in the IPA\nᶐ : Latin small letter alpha with retroflex hook\nⱯ ɐ : Turned A, which represents a near-open central vowel in the IPA\nΛ ʌ : Turned V (also called a wedge, a caret, or a hat), which represents an open-mid back unrounded vowel in the IPA\nⱰ ɒ : Turned alpha / script A, which represents an open back rounded vowel in the IPA\nᶛ : Modifier letter small turned alpha\nᴀ : Small capital A, an obsolete or non-standard symbol in the International Phonetic Alphabet used to represent various sounds (mainly open vowels)\nA a ᵄ : Modifier letters are used in the Uralic Phonetic Alphabet (UPA) (sometimes encoded with Unicode subscripts and superscripts)\na : Subscript small a is used in Indo-European studies\nꬱ : Small letter a reversed-schwa is used in the Teuthonista phonetic transcription system\nꞺ ꞻ : Glottal A, used in the transliteration of Ugaritic\n\nDerived signs, symbols and abbreviations\nª : an ordinal indicator\nÅ : Ångström sign\n∀ : a turned capital letter A, used in predicate logic to specify universal quantification ("for all")\n@ : At sign\n₳ : Argentine austral\n\nAncestors and siblings in other alphabets\n\n𐤀 : Semitic letter Aleph, from which the following symbols originally derive\nΑ α : Greek letter Alpha, from which the following letters derive\nА а : Cyrillic letter A\n : Coptic letter Alpha\n𐌀 : Old Italic A, which is the ancestor of modern Latin A\n : Runic letter ansuz, which probably derives from old Italic A\n : Gothic letter aza/asks\nԱ ա : Armenian letter Ayb\n\nComputing codes\n\n 1\n\nOther representations\n\nNotes\n\nFootnotes\n\nReferences\n\nExternal links\n\n History of the Alphabet\n \n\nISO basic Latin letters\nVowel letters', 'Alabama () is a state in the Southeastern region of the United States, bordered by Tennessee to the north; Georgia to the east; Florida and the Gulf of Mexico to the south; and Mississippi to the west. Alabama is the 30th largest by area and the 24th-most populous of the U.S. states. With a total of  of inland waterways, Alabama has among the most of any state.\n\nAlabama is nicknamed the Yellowhammer State, after the state bird. Alabama is also known as the "Heart of Dixie" and the "Cotton State". The state tree is the longleaf pine, and the state flower is the camellia. Alabama\'s capital is Montgomery, and its largest city by population and area is Huntsville. Its oldest city is Mobile, founded by French colonists in 1702 as the capital of French Louisiana. Greater Birmingham is Alabama\'s largest metropolitan area and its economic center.\n\nOriginally home to many native tribes, present-day Alabama was a Spanish territory beginning in the sixteenth century until the French acquired it in the early eighteenth century. The British won the territory in 1763 until losing it in the American Revolutionary War. Spain held Mobile as part of Spanish West Florida until 1813. In December 1819, Alabama was recognized as a state. During the antebellum period, Alabama was a major producer of cotton, and widely used African American slave labor. In 1861, the state seceded from the United States to become part of the Confederate States of America, with Montgomery acting as its first capital, and rejoined the Union in 1868. Following the American Civil War, Alabama would suffer decades of economic hardship, in part due to agriculture and a few cash crops being the main driver of the states economy. Similar to other former slave states, Alabamian legislators employed Jim Crow laws to disenfranchise and discriminate against African Americans from the late 19th century up until the 1960s. \n\nIn the early 20th century, despite the growth of major industries and urban centers, white rural interests dominated the state legislature through the mid-20th century. During this time, urban interests and African Americans were markedly under-represented. High-profile events such as the Selma to Montgomery march made the state a major focal point of the civil rights movement in the 1950s and 1960s. During and after World War II, Alabama grew as the state\'s economy diversified with new industries. NASA\'s Marshall Space Flight Center in Huntsville would help Alabama\'s economic growth in the mid-to-late 20th century, by developing an aerospace industry. Alabama\'s economy in the 21st century is based on automotive, finance, tourism, manufacturing, aerospace, mineral extraction, healthcare, education, retail, and technology.\n\nThe state\'s geography is diverse, with the north dominated by the mountainous Tennessee Valley and the south by Mobile Bay, a historically significant port. Politically, as part of the Deep South, Alabama is predominantly a conservative state, and culturally is known for its Southern culture. Within Alabama, American football, particularly at the college level at schools such as the University of Alabama, Auburn University, Alabama A&M University, Alabama State University, Troy University, the University of South Alabama, and Jacksonville State University, play a major part of the state\'s culture.\n\nEtymology\nThe European-American naming of the Alabama River and state was derived from the Alabama people, a Muskogean-speaking tribe whose members lived just below the confluence of the Coosa and Tallapoosa rivers on the upper reaches of the river. In the Alabama language, the word for a person of Alabama lineage is  (or variously  or  in different dialects; the plural form is ). The suggestion that "Alabama" was borrowed from the Choctaw language is unlikely. The word\'s spelling varies significantly among historical sources. The first usage appears in three accounts of the Hernando de Soto expedition of 1540: Garcilaso de la Vega used , while the Knight of Elvas and Rodrigo Ranjel wrote Alibamu and Limamu, respectively, in transliterations of the term. As early as 1702, the French called the tribe the , with French maps identifying the river as . Other spellings of the name have included Alibamu, Alabamo, Albama, Alebamon, Alibama, Alibamou, Alabamu, Allibamou. and possibly Alabahmu. The use of state names derived from Native American languages is common in the U.S.; an estimated 27 states have names of Native American origin.\n\nSources disagree on the word\'s meaning. Some scholars suggest the word comes from the Choctaw  (meaning \'plants\' or \'weeds\') and  (meaning \'to cut\', \'to trim\', or \'to gather\'). The meaning may have been \'clearers of the thicket\' or \'herb gatherers\', referring to clearing land for cultivation or collecting medicinal plants. The state has numerous place names of Native American origin. However, there are no correspondingly similar words in the Alabama language.\n\nAn 1842 article in the Jacksonville Republican proposed it meant \'Here We Rest\'. This notion was popularized in the 1850s through the writings of Alexander Beaufort Meek. Experts in the Muskogean languages have not found any evidence to support such a translation.\n\nHistory\n\nPre-European settlement\n\nIndigenous peoples of varying cultures lived in the area for thousands of years before the advent of European colonization. Trade with the northeastern tribes by the Ohio River began during the Burial Mound Period (1000BCE700CE) and continued until European contact.\n\nThe agrarian Mississippian culture covered most of the state from 1000 to 1600 CE, with one of its major centers built at what is now the Moundville Archaeological Site in Moundville, Alabama. This is the second-largest complex of the classic Middle Mississippian era, after Cahokia in present-day Illinois, which was the center of the culture. Analysis of artifacts from archaeological excavations at Moundville were the basis of scholars\' formulating the characteristics of the Southeastern Ceremonial Complex (SECC). Contrary to popular belief, the SECC appears to have no direct links to Mesoamerican culture, but developed independently. The Ceremonial Complex represents a major component of the religion of the Mississippian peoples; it is one of the primary means by which their religion is understood.\n\nAmong the historical tribes of Native American people living in present-day Alabama at the time of European contact were the Cherokee, an Iroquoian language people; and the Muskogean-speaking Alabama (Alibamu), Chickasaw, Choctaw, Creek, and Koasati. While part of the same large language family, the Muskogee tribes developed distinct cultures and languages.\n\nEuropean settlement\n\nThe Spanish were the first Europeans to reach Alabama during their exploration of North America in the 16th century. The expedition of Hernando de Soto passed through Mabila and other parts of the state in 1540. More than 160 years later, the French founded the region\'s first European settlement at Old Mobile in 1702. The city was moved to the current site of Mobile in 1711. This area was claimed by the French from 1702 to 1763 as part of La Louisiane.\n\nAfter the French lost to the British in the Seven Years\' War, it became part of British West Florida from 1763 to 1783. After the United States victory in the American Revolutionary War, the territory was divided between the United States and Spain. The latter retained control of this western territory from 1783 until the surrender of the Spanish garrison at Mobile to U.S. forces on April 13, 1813.\n\nThomas Bassett, a loyalist to the British monarchy during the Revolutionary era, was one of the earliest white settlers in the state outside Mobile. He settled in the Tombigbee District during the early 1770s. The district\'s boundaries were roughly limited to the area within a few miles of the Tombigbee River and included portions of what is today southern Clarke County, northernmost Mobile County, and most of Washington County.\n\nWhat is now the counties of Baldwin and Mobile became part of Spanish West Florida in 1783, part of the independent Republic of West Florida in 1810, and was finally added to the Mississippi Territory in 1812. Most of what is now the northern two-thirds of Alabama was known as the Yazoo lands beginning during the British colonial period. It was claimed by the Province of Georgia from 1767 onwards. Following the Revolutionary War, it remained a part of Georgia, although heavily disputed.\n\nWith the exception of the area around Mobile and the Yazoo lands, what is now the lower one-third of Alabama was made part of the Mississippi Territory when it was organized in 1798. The Yazoo lands were added to the territory in 1804, following the Yazoo land scandal. Spain kept a claim on its former Spanish West Florida territory in what would become the coastal counties until the Adams–Onís Treaty officially ceded it to the United States in 1819.\n\nEarly 19th century\n\nBefore Mississippi\'s admission to statehood on December 10, 1817, the more sparsely settled eastern half of the territory was separated and named the Alabama Territory. The United States Congress created the Alabama Territory on March 3, 1817. St. Stephens, now abandoned, served as the territorial capital from 1817 to 1819.\n\nAlabama was admitted as the 22nd state on December 14, 1819, with Congress selecting Huntsville as the site for the first Constitutional Convention. From July5 to August 2, 1819, delegates met to prepare the new state constitution. Huntsville served as temporary capital from 1819 to 1820, when the seat of government moved to Cahaba in Dallas County.\n\nCahaba, now a ghost town, was the first permanent state capital from 1820 to 1825. The Alabama Fever land rush was underway when the state was admitted to the Union, with settlers and land speculators pouring into the state to take advantage of fertile land suitable for cotton cultivation. Part of the frontier in the 1820s and 1830s, its constitution provided for universal suffrage for white men.\n\nSoutheastern planters and traders from the Upper South brought slaves with them as the cotton plantations in Alabama expanded. The economy of the central Black Belt (named for its dark, productive soil) was built around large cotton plantations whose owners\' wealth grew mainly from slave labor. The area also drew many poor, disenfranchised people who became subsistence farmers. Alabama had an estimated population of under 10,000 people in 1810, but it increased to more than 300,000 people by 1830. Most Native American tribes were completely removed from the state within a few years of the passage of the Indian Removal Act by Congress in 1830.\n\nFrom 1826 to 1846, Tuscaloosa served as Alabama\'s capital. On January 30, 1846, the Alabama legislature announced it had voted to move the capital city from Tuscaloosa to Montgomery. The first legislative session in the new capital met in December 1847. A new capitol building was erected under the direction of Stephen Decatur Button of Philadelphia. The first structure burned down in 1849, but was rebuilt on the same site in 1851. This second capitol building in Montgomery remains to the present day. It was designed by Barachias Holt of Exeter, Maine.\n\nCivil War and Reconstruction\n\nBy 1860, the population had increased to 964,201 people, of which nearly half, 435,080, were enslaved African Americans, and 2,690 were free people of color. On January 11, 1861, Alabama declared its secession from the Union. After remaining an independent republic for a few days, it joined the Confederate States of America. The Confederacy\'s capital was initially at Montgomery. Alabama was heavily involved in the American Civil War. Although comparatively few battles were fought in the state, Alabama contributed about 120,000 soldiers to the war effort.\n\nA company of cavalry soldiers from Huntsville, Alabama, joined Nathan Bedford Forrest\'s battalion in Hopkinsville, Kentucky. The company wore new uniforms with yellow trim on the sleeves, collar and coattails. This led to them being greeted with "Yellowhammer", and the name later was applied to all Alabama troops in the Confederate Army.\n\nAlabama\'s slaves were freed by the 13th Amendment in 1865. Alabama was under military rule from the end of the war in May 1865 until its official restoration to the Union in 1868. From 1867 to 1874, with most white citizens barred temporarily from voting and freedmen enfranchised, many African Americans emerged as political leaders in the state. Alabama was represented in Congress during this period by three African-American congressmen: Jeremiah Haralson, Benjamin S. Turner, and James T. Rapier.\n\nFollowing the war, the state remained chiefly agricultural, with an economy tied to cotton. During Reconstruction, state legislators ratified a new state constitution in 1868 which created the state\'s first public school system and expanded women\'s rights. Legislators funded numerous public road and railroad projects, although these were plagued with allegations of fraud and misappropriation. Organized insurgent, resistance groups tried to suppress the freedmen and Republicans. Besides the short-lived original Ku Klux Klan, these included the Pale Faces, Knights of the White Camellia, Red Shirts, and the White League.\n\nReconstruction in Alabama ended in 1874, when the Democrats regained control of the legislature and governor\'s office through an election dominated by fraud and violence. They wrote another constitution in 1875, and the legislature passed the Blaine Amendment, prohibiting public money from being used to finance religious-affiliated schools. The same year, legislation was approved that called for racially segregated schools. Railroad passenger cars were segregated in 1891.\n\n20th century\n\nThe new 1901 Constitution of Alabama included provisions for voter registration that effectively disenfranchised large portions of the population, including nearly all African Americans and Native Americans, and tens of thousands of poor European Americans, through making voter registration difficult, requiring a poll tax and literacy test. The 1901 constitution required racial segregation of public schools. By 1903 only 2,980 African Americans were registered in Alabama, although at least 74,000 were literate. This compared to more than 181,000 African Americans eligible to vote in 1900. The numbers dropped even more in later decades. The state legislature passed additional racial segregation laws related to public facilities into the 1950s: jails were segregated in 1911; hospitals in 1915; toilets, hotels, and restaurants in 1928; and bus stop waiting rooms in 1945.\n\nWhile the planter class had persuaded poor whites to vote for this legislative effort to suppress black voting, the new restrictions resulted in their disenfranchisement as well, due mostly to the imposition of a cumulative poll tax. By 1941, whites constituted a slight majority of those disenfranchised by these laws: 600,000 whites vs. 520,000 African-Americans. Nearly all Blacks had lost the ability to vote. Despite numerous legal challenges which succeeded in overturning certain provisions, the state legislature would create new ones to maintain disenfranchisement. The exclusion of blacks from the political system persisted until after passage of federal civil rights legislation in 1965 to enforce their constitutional rights as citizens.\n\nThe rural-dominated Alabama legislature consistently underfunded schools and services for the disenfranchised African Americans, but it did not relieve them of paying taxes. Partially as a response to chronic underfunding of education for African Americans in the South, the Rosenwald Fund began funding the construction of what came to be known as Rosenwald Schools. In Alabama these schools were designed and the construction partially financed with Rosenwald funds, which paid one-third of the construction costs. The fund required the local community and state to raise matching funds to pay the rest. Black residents effectively taxed themselves twice, by raising additional monies to supply matching funds for such schools, which were built in many rural areas. They often donated land and labor as well.\n\nBeginning in 1913, the first 80 Rosenwald Schools were built in Alabama for African-American children. A total of 387 schools, seven teachers\' houses, and several vocational buildings were completed by 1937 in the state. Several of the surviving school buildings in the state are now listed on the National Register of Historic Places.\n\nContinued racial discrimination and lynchings, agricultural depression, and the failure of the cotton crops due to boll weevil infestation led tens of thousands of African Americans from rural Alabama and other states to seek opportunities in northern and midwestern cities during the early decades of the 20th century as part of the Great Migration out of the South. Reflecting this emigration, the population growth rate in Alabama (see "historical populations" table below) dropped by nearly half from 1910 to 1920.\n\nAt the same time, many rural people migrated to the city of Birmingham to work in new industrial jobs. Birmingham experienced such rapid growth it was called the "Magic City". By 1920, Birmingham was the 36th-largest city in the United States. Heavy industry and mining were the basis of its economy. Its residents were under-represented for decades in the state legislature, which refused to redistrict after each decennial census according to population changes, as it was required by the state constitution. This did not change until the late 1960s following a lawsuit and court order.\n\nIndustrial development related to the demands of World War II brought a level of prosperity to the state not seen since before the civil war. Rural workers poured into the largest cities in the state for better jobs and a higher standard of living. One example of this massive influx of workers occurred in Mobile. Between 1940 and 1943, more than 89,000 people moved into the city to work for war-related industries. Cotton and other cash crops faded in importance as the state developed a manufacturing and service base.\n\nDespite massive population changes in the state from 1901 to 1961, the rural-dominated legislature refused to reapportion House and Senate seats based on population, as required by the state constitution to follow the results of decennial censuses. They held on to old representation to maintain political and economic power in agricultural areas. One result was that Jefferson County, containing Birmingham\'s industrial and economic powerhouse, contributed more than one-third of all tax revenue to the state, but did not receive a proportional amount in services. Urban interests were consistently underrepresented in the legislature. A 1960 study noted that because of rural domination, "a minority of about 25% of the total state population is in majority control of the Alabama legislature."\n\nIn the United States Supreme Court cases of Baker v. Carr (1962) and Reynolds v. Sims (1964), the court ruled that the principle of "one man, one vote" needed to be the basis of both houses of state legislatures, and that their districts had to be based on population rather than geographic counties.\n\nIn 1972, for the first time since 1901, the legislature completed the congressional redistricting based on the decennial census. This benefited the urban areas that had developed, as well as all in the population who had been underrepresented for more than sixty years. Other changes were made to implement representative state house and senate districts.\n\nAfrican Americans continued to press in the 1950s and 1960s to end disenfranchisement and segregation in the state through the civil rights movement, including legal challenges. In 1954, the U.S. Supreme Court ruled in Brown v. Board of Education that public schools had to be desegregated, but Alabama was slow to comply. During the 1960s, under Governor George Wallace, Alabama resisted compliance with federal demands for desegregation. The civil rights movement had notable events in Alabama, including the Montgomery bus boycott (1955–1956), Freedom Rides in 1961, and 1965 Selma to Montgomery marches. These contributed to Congressional passage and enactment of the Civil Rights Act of 1964 and Voting Rights Act of 1965 by the U.S. Congress.\n\nLegal segregation ended in the states in 1964, but Jim Crow customs often continued until specifically challenged in court. According to The New York Times, by 2017, many of Alabama\'s African-Americans were living in Alabama\'s cities such as Birmingham and Montgomery. Also, the Black Belt region across central Alabama "is home to largely poor counties that are predominantly African-American. These counties include Dallas, Lowndes, Marengo and Perry."\n\nAlabama has made some changes since the late 20th century and has used new types of voting to increase representation. In the 1980s, an omnibus redistricting case, Dillard v. Crenshaw County, challenged the at-large voting for representative seats of 180 Alabama jurisdictions, including counties and school boards. At-large voting had diluted the votes of any minority in a county, as the majority tended to take all seats. Despite African Americans making up a significant minority in the state, they had been unable to elect any representatives in most of the at-large jurisdictions.\n\nAs part of settlement of this case, five Alabama cities and counties, including Chilton County, adopted a system of cumulative voting for election of representatives in multi-seat jurisdictions. This has resulted in more proportional representation for voters. In another form of proportional representation, 23 jurisdictions use limited voting, as in Conecuh County. In 1982, limited voting was first tested in Conecuh County. Together use of these systems has increased the number of African Americans and women being elected to local offices, resulting in governments that are more representative of their citizens.\n\nBeginning in the 1960s, the state\'s economy shifted away from its traditional lumber, steel, and textile industries because of increased foreign competition. Steel jobs, for instance, declined from 46,314 in 1950 to 14,185 in 2011. However, the state, particularly Huntsville, benefited from the opening of the George C. Marshall Space Flight Center in 1960, a major facility in the development of the Saturn rocket program and the space shuttle. Technology and manufacturing industries, such as automobile assembly, replaced some the state\'s older industries in the late twentieth century, but the state\'s economy and growth lagged behind other states in the area, such as Georgia and Florida.\n\n21st century\n\nIn 2001, Alabama Supreme Court chief justice Roy Moore installed a statue of the Ten Commandments in the capitol in Montgomery. In 2002, the 11th US Circuit Court ordered the statue removed, but Moore refused to follow the court order, which led to protests around the capitol in favor of keeping the monument.  The monument was removed in August 2003.\n\nA few natural disasters have occurred in the state in the twenty-first century. In 2004, Hurricane Ivan, a category 3 storm upon landfall, struck the state and caused over $18 billion of damage. It was among the most destructive storms to strike the state in its modern history. A super outbreak of 62 tornadoes hit the state in April 2011 and killed 238 people, devastating many communities.\n\nGeography\n\nAlabama is the thirtieth-largest state in the United States with  of total area: 3.2% of the area is water, making Alabama 23rd in the amount of surface water, also giving it the second-largest inland waterway system in the United States. About three-fifths of the land area is part of the Gulf Coastal Plain, a gentle plain with a general descent towards the Mississippi River and the Gulf of Mexico. The North Alabama region is mostly mountainous, with the Tennessee River cutting a large valley and creating numerous creeks, streams, rivers, mountains, and lakes.\n\nAlabama is bordered by the states of Tennessee to the north, Georgia to the east, Florida to the south, and Mississippi to the west. Alabama has coastline at the Gulf of Mexico, in the extreme southern edge of the state. The state ranges in elevation from sea level at Mobile Bay to more than  in the northeast, to Mount Cheaha at .\n\nAlabama\'s land consists of  of forest or 67% of the state\'s total land area. Suburban Baldwin County, along the Gulf Coast, is the largest county in the state in both land area and water area.\n\nAreas in Alabama administered by the National Park Service include Horseshoe Bend National Military Park near Alexander City; Little River Canyon National Preserve near Fort Payne; Russell Cave National Monument in Bridgeport; Tuskegee Airmen National Historic Site in Tuskegee; and Tuskegee Institute National Historic Site near Tuskegee. Additionally, Alabama has four National Forests: Conecuh, Talladega, Tuskegee, and William B. Bankhead. Alabama also contains the Natchez Trace Parkway, the Selma To Montgomery National Historic Trail, and the Trail of Tears National Historic Trail.\n\nNotable natural wonders include: the "Natural Bridge" rock, the longest natural bridge east of the Rockies, located just south of Haleyville; Cathedral Caverns in Marshall County, named for its cathedral-like appearance, features one of the largest cave entrances and stalagmites in the world; Ecor Rouge in Fairhope, the highest coastline point between Maine and Mexico; DeSoto Caverns in Childersburg, the first officially recorded cave in the United States; Noccalula Falls in Gadsden features a 90-foot waterfall; Dismals Canyon near Phil Campbell, home to two waterfalls, six natural bridges and allegedly served as a hideout for legendary outlaw Jesse James; Stephens Gap Cave in Jackson County boasts a 143-foot pit, two waterfalls and is one of the most photographed wild cave scenes in America; Little River Canyon near Fort Payne, one of the nation\'s longest mountaintop rivers; Rickwood Caverns near Warrior features an underground pool, blind cave fish and 260-million-year-old limestone formations; and the Walls of Jericho canyon on the Alabama-Tennessee state line.\n\nA -wide meteorite impact crater is located in Elmore County, just north of Montgomery. This is the Wetumpka crater, the site of "Alabama\'s greatest natural disaster". A -wide meteorite hit the area about 80 million years ago. The hills just east of downtown Wetumpka showcase the eroded remains of the impact crater that was blasted into the bedrock, with the area labeled the Wetumpka crater or astrobleme ("star-wound") because of the concentric rings of fractures and zones of shattered rock that can be found beneath the surface. In 2002, Christian Koeberl with the Institute of Geochemistry University of Vienna published evidence and established the site as the 157th recognized impact crater on Earth.\n\nClimate\n\nThe state is classified as humid subtropical (Cfa) under the Koppen Climate Classification. The average annual temperature is 64°F (18°C). Temperatures tend to be warmer in the southern part of the state with its proximity to the Gulf of Mexico, while the northern parts of the state, especially in the Appalachian Mountains in the northeast, tend to be slightly cooler. Generally, Alabama has very hot summers and mild winters with copious precipitation throughout the year. Alabama receives an average of  of rainfall annually and enjoys a lengthy growing season of up to 300 days in the southern part of the state.\n\nSummers in Alabama are among the hottest in the U.S., with high temperatures averaging over  throughout the summer in some parts of the state. Alabama is also prone to tropical storms and hurricanes. Areas of the state far away from the Gulf are not immune to the effects of the storms, which often dump tremendous amounts of rain as they move inland and weaken.\n\nSouth Alabama reports many thunderstorms. The Gulf Coast, around Mobile Bay, averages between 70 and 80 days per year with thunder reported. This activity decreases somewhat further north in the state, but even the far north of the state reports thunder on about 60 days per year. Occasionally, thunderstorms are severe with frequent lightning and large hail; the central and northern parts of the state are most vulnerable to this type of storm. Alabama ranks ninth in the number of deaths from lightning and tenth in the number of deaths from lightning strikes per capita.\n\nAlabama, along with Oklahoma and Iowa, has the most confirmed F5 and EF5 tornadoes of any state, according to statistics from the National Climatic Data Center for the period January 1, 1950, to June 2013. Several long-tracked F5/EF5 tornadoes have contributed to Alabama reporting more tornado fatalities since 1950 than any other state. The state was affected by the 1974 Super Outbreak and was devastated tremendously by the 2011 Super Outbreak. The 2011 Super Outbreak produced a record amount of tornadoes in the state. The tally reached 62.\n\nThe peak season for tornadoes varies from the northern to southern parts of the state. Alabama is one of the few places in the world that has a secondary tornado season in November and December besides the typically severe spring. The northern part—along the Tennessee River Valley—is most vulnerable. The area of Alabama and Mississippi most affected by tornadoes is sometimes referred to as Dixie Alley, as distinct from the Tornado Alley of the Southern Plains.\n\nWinters are generally mild in Alabama, as they are throughout most of the Southeastern United States, with average January low temperatures around  in Mobile and around  in Birmingham. Although snow is a rare event in much of Alabama, areas of the state north of Montgomery may receive a dusting of snow a few times every winter, with an occasional moderately heavy snowfall every few years. Historic snowfall events include New Year\'s Eve 1963 snowstorm and the 1993 Storm of the Century. The annual average snowfall for the Birmingham area is  per year. In the southern Gulf coast, snowfall is less frequent, sometimes going several years without any snowfall.\n\nAlabama\'s highest temperature of  was recorded on September 5, 1925, in the unincorporated community of Centerville. The record low of  occurred on January 30, 1966, in New Market.\n\nFlora and fauna\n\nAlabama is home to a diverse array of flora and fauna in habitats that range from the Tennessee Valley, Appalachian Plateau, and Ridge-and-Valley Appalachians of the north to the Piedmont, Canebrake, and Black Belt of the central region to the Gulf Coastal Plain and beaches along the Gulf of Mexico in the south. The state is usually ranked among the top in nation for its range of overall biodiversity.\n\nAlabama is in the subtropical coniferous forest biome and once boasted huge expanses of pine forest, which still form the largest proportion of forests in the state. It currently ranks fifth in the nation for the diversity of its flora. It is home to nearly 4,000 pteridophyte and spermatophyte plant species.\n\nIndigenous animal species in the state include 62 mammal species, 93 reptile species, 73 amphibian species, roughly 307 native freshwater fish species, and 420 bird species that spend at least part of their year within the state. Invertebrates include 97 crayfish species and 383 mollusk species. 113 of these mollusk species have never been collected outside the state.\n\nCensus-designated and metropolitan areas\n\nCities\n\nDemographics\n\nAccording to the 2020 United States census the population of Alabama was 5,024,279 on April 1, 2020, which represents an increase of 244,543 or 5.12%, since the 2010 census. This includes a natural increase since the last census of 121,054 (502,457 births minus 381,403 deaths) and an increase due to net migration of 104,991 into the state.\n\nImmigration from outside the U.S. resulted in a net increase of 31,180 people, and migration within the country produced a net gain of 73,811 people. The state had 108,000 foreign-born (2.4% of the state population), of which an estimated 22.2% were undocumented (24,000).\n\nThe center of population of Alabama is located in Chilton County, outside the town of Jemison.\n\nAncestry\n\nThose citing "American" ancestry in Alabama are of overwhelmingly English extraction, however most English Americans identify simply as having American ancestry because their roots have been in North America for so long, in many cases since the early sixteen hundreds.  Demographers estimate that a minimum of 20–23% of people in Alabama are of predominantly English ancestry and state that the figure is probably much higher. In the 1980 census 1,139,976 people in Alabama cited that they were of English ancestry out of a total state population of 2,824,719 making them 41% of the state at the time and the largest ethnic group.\n\nIn 2011, 46.6% of Alabama\'s population younger than age1 were minorities. The largest reported ancestry groups in Alabama are American (13.4%), Irish (10.5%), English (10.2%), German (7.9%), and Scots-Irish (2.5%) based on 2006-2008 Census data.\n\nThe Scots-Irish were the largest non-English immigrant group from the British Isles before the American Revolution, and many settled in the South, later moving into the Deep South as it was developed.\n\nIn 1984, under the Davis–Strong Act, the state legislature established the Alabama Indian Affairs Commission. Native American groups within the state had increasingly been demanding recognition as ethnic groups and seeking an end to discrimination. Given the long history of slavery and associated racial segregation, the Native American peoples, who have sometimes been of mixed race, have insisted on having their cultural identification respected. In the past, their self-identification was often overlooked as the state tried to impose a binary breakdown of society into white and black. The state has officially recognized nine American Indian tribes in the state, descended mostly from the Five Civilized Tribes of the American Southeast. These are the following.\n Poarch Band of Creek Indians (who also have federal recognition)\n MOWA Band of Choctaw Indians\n Star Clan of Muscogee Creeks\n Echota Cherokee Tribe of Alabama\n Cherokee Tribe of Northeast Alabama\n Cher-O-Creek Intra Tribal Indians\n Ma-Chis Lower Creek Indian Tribe\n Piqua Shawnee Tribe\n Ani-Yun-Wiya Nation\n\nThe state government has promoted recognition of Native American contributions to the state, including the designation in 2000 for Columbus Day to be jointly celebrated as American Indian Heritage Day.\n\nLanguage\nMost Alabama residents (95.1% of those five and older) spoke only English at home in 2010, a minor decrease from 96.1% in 2000. Alabama English is predominantly Southern, and is related to South Midland speech which was taken across the border from Tennessee. In the major Southern speech region, there is the decreasing loss of the final r, for example the "boyd" pronunciation of "bird". In the northern third of the state, there is a South Midland "arm" and "barb" rhyming with "form" and "orb". Unique words in Alabama English include: redworm (earthworm), peckerwood (woodpecker), snake doctor and snake feeder (dragonfly), tow sack (burlap bag), plum peach (clingstone), French harp (harmonica), and dog irons (andirons).\n\nReligion\n\nIn the 2008 American Religious Identification Survey, 86% of Alabama respondents reported their religion as Christian, including 6% Catholic, with 11% as having no religion. The composition of other traditions is 0.5% Mormon, 0.5% Jewish, 0.5% Muslim, 0.5% Buddhist, and 0.5% Hindu.\n\nAlabama is located in the middle of the Bible Belt, a region of numerous Protestant Christians. Alabama has been identified as one of the most religious states in the United States, with about 58% of the population attending church regularly. A majority of people in the state identify as Evangelical Protestant. , the three largest denominational groups in Alabama are the Southern Baptist Convention, The United Methodist Church, and non-denominational Evangelical Protestant.\n\nIn Alabama, the Southern Baptist Convention has the highest number of adherents with 1,380,121; this is followed by the United Methodist Church with 327,734 adherents, non-denominational Evangelical Protestant with 220,938 adherents, and the Catholic Church with 150,647 adherents. Many Baptist and Methodist congregations became established in the Great Awakening of the early 19th century, when preachers proselytized across the South. The Assemblies of God had almost 60,000 members, the Churches of Christ had nearly 120,000 members. The Presbyterian churches, strongly associated with Scots-Irish immigrants of the 18th century and their descendants, had a combined membership around 75,000 (PCA—28,009 members in 108 congregations, PC(USA)—26,247 members in 147 congregations, the Cumberland Presbyterian Church—6,000 members in 59 congregations, the Cumberland Presbyterian Church in America—5,000 members and fifty congregations plus the EPC and Associate Reformed Presbyterians with 230 members and nine congregations).\n\nIn a 2007 survey, nearly 70% of respondents could name all four of the Christian Gospels. Of those who indicated a religious preference, 59% said they possessed a "full understanding" of their faith and needed no further learning. In a 2007 poll, 92% of Alabamians reported having at least some confidence in churches in the state.\n\nAlthough in much smaller numbers, many other religious faiths are represented in the state as well, including Judaism, Islam, Hinduism, Buddhism, Sikhism, the Baháʼí Faith, and Unitarian Universalism.\n\nJews have been present in what is now Alabama since 1763, during the colonial era of Mobile, when Sephardic Jews immigrated from London. The oldest Jewish congregation in the state is Congregation Sha\'arai Shomayim in Mobile. It was formally recognized by the state legislature on January 25, 1844. Later immigrants in the nineteenth and twentieth centuries tended to be Ashkenazi Jews from eastern Europe. Jewish denominations in the state include two Orthodox, four Conservative, ten Reform, and one Humanistic synagogue.\n\nMuslims have been increasing in Alabama, with 31 mosques built by 2011, many by African-American converts.\n\nSeveral Hindu temples and cultural centers in the state have been founded by Indian immigrants and their descendants, the best-known being the Shri Swaminarayan Mandir in Birmingham, the Hindu Temple and Cultural Center of Birmingham in Pelham, the Hindu Cultural Center of North Alabama in Capshaw, and the Hindu Mandir and Cultural Center in Tuscaloosa.\n\nThere are six Dharma centers and organizations for Theravada Buddhists. Most monastic Buddhist temples are concentrated in southern Mobile County, near Bayou La Batre. This area has attracted an influx of refugees from Cambodia, Laos, and Vietnam during the 1970s and thereafter. The four temples within a ten-mile radius of Bayou La Batre, include Chua Chanh Giac, Wat Buddharaksa, and Wat Lao Phoutthavihan.\n\nThe first community of adherents of the Baháʼí Faith in Alabama was founded in 1896 by Paul K. Dealy, who moved from Chicago to Fairhope. Baháʼí centers in Alabama exist in Birmingham, Huntsville, and Florence.\n\nHealth\nIn 2018, life expectancy in Alabama was 75.1 years, below the national average of 78.7 years and is the third lowest life expectancy in the country. Factors that can cause lower life expectancy are maternal mortality, suicide, and gun crimes.\n\nA Centers for Disease Control and Prevention study in 2008 showed that obesity in Alabama is a problem, with most counties having more than 29% of adults obese, except for ten which had a rate between 26% and 29%. Residents of the state, along with those in five other states, were least likely in the nation to be physically active during leisure time. Alabama, and the southeastern U.S. in general, has one of the highest incidences of adult onset diabetes in the country, exceeding 10% of adults.\n\nOn May 14, 2019, Alabama passed the Human Life Protection Act, banning abortion at any stage of pregnancy unless there is a "serious health risk", with no exceptions for rape and incest. The law, if enacted, would punish doctors who perform abortions with 10 to 99 years imprisonment and be the most restrictive abortion law in the country. However, on October 29, 2019, U.S. District Judge Myron Thompson blocked the law from taking effect.\n\nEconomy\n\nThe state has invested in aerospace, education, health care, banking, and various heavy industries, including automobile manufacturing, mineral extraction, steel production and fabrication. By 2006, crop and animal production in Alabama was valued at $1.5billion. In contrast to the primarily agricultural economy of the previous century, this was only about one percent of the state\'s gross domestic product. The number of private farms has declined at a steady rate since the 1960s, as land has been sold to developers, timber companies, and large farming conglomerates.\n\nNon-agricultural employment in 2008 was 121,800 in management occupations; 71,750 in business and financial operations; 36,790 in computer-related and mathematical occupation; 44,200 in architecture and engineering; 12,410 in life, physical, and social sciences; 32,260 in community and social services; 12,770 in legal occupations; 116,250 in education, training, and library services; 27,840 in art, design and media occupations; 121,110 in healthcare; 44,750 in fire fighting, law enforcement, and security; 154,040 in food preparation and serving; 76,650 in building and grounds cleaning and maintenance; 53,230 in personal care and services; 244,510 in sales; 338,760 in office and administration support; 20,510 in farming, fishing, and forestry; 120,155 in construction and mining, gas, and oil extraction; 106,280 in installation, maintenance, and repair; 224,110 in production; and 167,160 in transportation and material moving.\n\nAccording to the U.S. Bureau of Economic Analysis, the 2008 total gross state product was $170billion, or $29,411 per capita. Alabama\'s 2012 GDP increased 1.2% from the previous year. The single largest increase came in the area of information. In 2010, per capita income for the state was $22,984.\n\nThe state\'s seasonally adjusted unemployment rate was 5.8% in April 2015. This compared to a nationwide seasonally adjusted rate of 5.4%.\n\nAlabama has no minimum wage and in February 2016 passed legislation preventing municipalities from setting one. (A Birmingham city ordinance would have raised theirs to $10.10.)\n\n, Alabama has the sixth highest poverty rate among states in the U.S. In 2017, United Nations Special Rapporteur Philip Alston toured parts of rural Alabama and observed environmental conditions he said were poorer than anywhere he had seen in the developed world.\n\nLargest employers\n\nThe five employers that employed the most employees in Alabama in April 2011 were:\n\nThe next twenty largest employers, , included:\n\nAgriculture\nAlabama\'s agricultural outputs include poultry and eggs, cattle, fish, plant nursery items, peanuts, cotton, grains such as corn and sorghum, vegetables, milk, soybeans, and peaches. Although known as "The Cotton State", Alabama ranks between eighth and tenth in national cotton production, according to various reports, with Texas, Georgia and Mississippi comprising the top three.\n\nAquaculture \n\nAquaculture is a large part of the economy of Alabama. Alabamians began to practice aquaculture in the early 1960s. U.S. farm-raised catfish is the 8th most popular seafood product in America. By 2008, approximately 4,000 people in Alabama were employed by the catfish industry and Alabama produced 132 million pounds of catfish. In 2020, Alabama produced ⅓ of the United States\' farm-raised catfish. The total 2020 sales of catfish raised in Alabama equaled $307 million but by 2020 the total employment of Alabamians fell to 2,442.\n\nFrom the early 2000s to 2020, the Alabamian catfish industry has declined from 250 farms and 4 processors to 66 farms and 2 processors. Reasons for this decline include increased feed prices, catfish alternatives, COVID-19’s impact on restaurant sales, disease, and fish size.\n\nIndustry\nAlabama\'s industrial outputs include iron and steel products (including cast-iron and steel pipe); paper, lumber, and wood products; mining (mostly coal); plastic products; cars and trucks; and apparel. In addition, Alabama produces aerospace and electronic products, mostly in the Huntsville area, the location of NASA\'s George C. Marshall Space Flight Center and the U.S. Army Materiel Command, headquartered at Redstone Arsenal.\n\nA great deal of Alabama\'s economic growth since the 1990s has been due to the state\'s expanding automotive manufacturing industry. Located in the state are Honda Manufacturing of Alabama, Hyundai Motor Manufacturing Alabama, Mercedes-Benz U.S. International, and Toyota Motor Manufacturing Alabama, as well as their various suppliers. Since 1993, the automobile industry has generated more than 67,800 new jobs in the state. Alabama currently ranks 4th in the nation for vehicle exports.\n\nAutomakers accounted for approximately a third of the industrial expansion in the state in 2012. The eight models produced at the state\'s auto factories totaled combined sales of 74,335 vehicles for 2012. The strongest model sales during this period were the Hyundai Elantra compact car, the Mercedes-Benz GL-Class sport utility vehicle and the Honda Ridgeline sport utility truck.\n\nSteel producers Outokumpu, Nucor, SSAB, ThyssenKrupp, and U.S. Steel have facilities in Alabama and employ more than 10,000 people. In May 2007, German steelmaker ThyssenKrupp selected Calvert in Mobile County for a 4.65billion combined stainless and carbon steel processing facility. ThyssenKrupp\'s stainless steel division, Inoxum, including the stainless portion of the Calvert plant, was sold to Finnish stainless steel company Outokumpu in 2012. The remaining portion of the ThyssenKrupp plant had final bids submitted by ArcelorMittal and Nippon Steel for $1.6billion in March 2013. Companhia Siderúrgica Nacional submitted a combined bid for the mill at Calvert, plus a majority stake in the ThyssenKrupp mill in Brazil, for $3.8billion. In July 2013, the plant was sold to ArcelorMittal and Nippon Steel.\n\nThe Hunt Refining Company, a subsidiary of Hunt Consolidated, Inc., is based in Tuscaloosa and operates a refinery there. The company also operates terminals in Mobile, Melvin, and Moundville. JVC America, Inc. operates an optical disc replication and packaging plant in Tuscaloosa.\n\nThe Goodyear Tire and Rubber Company operates a large plant in Gadsden which employs about 1,400 people. It has been in operation since 1929.\n\nConstruction of an Airbus A320 family aircraft assembly plant in Mobile was formally announced by Airbus CEO Fabrice Brégier from the Mobile Convention Center on July 2, 2012. The plans include a $600million factory at the Brookley Aeroplex for the assembly of the A319, A320 and A321 aircraft. Construction began in 2013, with plans for it to become operable by 2015 and produce up to 50 aircraft per year by 2017. The assembly plant is the company\'s first factory to be built within the United States. It was announced on February 1, 2013, that Airbus had hired Alabama-based Hoar Construction to oversee construction of the facility.\n\nTourism and entertainment\n\nAccording to Business Insider, Alabama ranked 14th in most popular states to visit in 2014. An estimated 26 million tourists visited the state in 2017 and spent $14.3 billion, providing directly or indirectly 186,900 jobs in the state, which includes 362,000 International tourists spending $589 million.\n\nThe state is home to various attractions, natural features, parks and events that attract visitors from around the globe, notably the annual Hangout Music Festival, held on the public beaches of Gulf Shores; the Alabama Shakespeare Festival, one of the ten largest Shakespeare festivals in the world; the Robert Trent Jones Golf Trail, a collection of championship caliber golf courses distributed across the state; casinos such as Victoryland; amusement parks such as Alabama Splash Adventure; the Riverchase Galleria, one of the largest shopping centers in the southeast; Guntersville Lake, voted the best lake in Alabama by Southern Living Magazine readers; and the Alabama Museum of Natural History, the oldest museum in the state.\n\nMobile is known for having the oldest organized Mardi Gras celebration in the United States, beginning in 1703. It was also host to the first formally organized Mardi Gras parade in the United States in 1830, a tradition that continues to this day. Mardi Gras is an official state holiday in Mobile and Baldwin counties.\n\nIn 2018, Mobile\'s Mardi Gras parade was the state\'s top event, producing the most tourists with an attendance of 892,811. The top attraction was the U.S. Space & Rocket Center in Huntsville with an attendance of 849,981, followed by the Birmingham Zoo with 543,090. Of the parks and natural destinations, Alabama\'s Gulf Coast topped the list with 6,700,000 visitors.\n\nAlabama has historically been a popular region for film shoots due to its diverse landscapes and contrast of environments. Movies filmed in Alabama include: Close Encounters of the Third Kind, Get Out, 42, Selma, Big Fish, The Final Destination, Due Date, Need For Speed and many more.\n\nHealthcare\nUAB Hospital, USA Health University Hospital, Huntsville Hospital, and Children\'s Hospital of Alabama are the only LevelI trauma centers in Alabama. UAB is the largest state government employer in Alabama, with a workforce of about 18,000. A 2017 study found that Alabama had the least competitive health insurance market in the country, with Blue Cross and Blue Shield of Alabama having a market share of 84% followed by UnitedHealth Group at 7%.\n\nBanking\n\nRegions Financial Corporation is the largest bank headquartered in or operating in Alabama. PNC Financial Services  and Wells Fargo also have a major presence in Alabama.\n\nWells Fargo has a regional headquarters, an operations center campus, and a $400million data center in Birmingham. Many smaller banks are also headquartered in the Birmingham area, including ServisFirst and New South Federal Savings Bank. Birmingham also serves as the headquarters for several large investment management companies, including Harbert Management Corporation.\n\nElectronics and communications\nTelecommunications provider AT&T, formerly BellSouth, has a major presence in Alabama with several large offices in Birmingham.\n\nMany technology companies are headquartered in Huntsville, such as ADTRAN, a network access company; Intergraph, a computer graphics company; and Avocent, an IT infrastructure company.\n\nConstruction\nBrasfield & Gorrie, BE&K, Hoar Construction, and B.L. Harbert International, based in Alabama and subsidiaries of URS Corporation, are all routinely are included in the Engineering News-Record lists of top design, international construction, and engineering firms.\n\nLaw and government\n\nState government\n\nThe foundational document for Alabama\'s government is the Alabama Constitution, which was ratified in 1901. With over 850 amendments and almost 87,000 words, it is by some accounts the world\'s longest constitution and is roughly forty times the length of the United States Constitution.\n\nThere has been a significant movement to rewrite and modernize Alabama\'s constitution. Critics argue that Alabama\'s constitution maintains highly centralized power with the state legislature, leaving practically no power in local hands. Most counties do not have home rule. Any policy changes proposed in different areas of the state must be approved by the entire Alabama legislature and, frequently, by state referendum. One criticism of the current constitution claims that its complexity and length intentionally codify segregation and racism.\n\nAlabama\'s government is divided into three coequal branches. The legislative branch is the Alabama Legislature, a bicameral assembly composed of the Alabama House of Representatives, with 105 members, and the Alabama Senate, with 35 members. The Legislature is responsible for writing, debating, passing, or defeating state legislation. The Republican Party currently holds a majority in both houses of the Legislature. The Legislature has the power to override a gubernatorial veto by a simple majority (most state Legislatures require a two-thirds majority to override a veto).\n\nUntil 1964, the state elected state senators on a geographic basis by county, with one per county. It had not redistricted congressional districts since passage of its constitution in 1901; as a result, urbanized areas were grossly underrepresented. It had not changed legislative districts to reflect the decennial censuses, either. In Reynolds v. Sims (1964), the U.S. Supreme Court implemented the principle of "one man, one vote", ruling that congressional districts had to be reapportioned based on censuses (as the state already included in its constitution but had not implemented.) Further, the court ruled that both houses of bicameral state legislatures had to be apportioned by population, as there was no constitutional basis for states to have geographically based systems.\n\nAt that time, Alabama and many other states had to change their legislative districting, as many across the country had systems that underrepresented urban areas and districts. This had caused decades of underinvestment in such areas. For instance, Birmingham and Jefferson County taxes had supplied one-third of the state budget, but Jefferson County received only 1/67th of state services in funding. Through the legislative delegations, the Alabama legislature kept control of county governments.\n\nThe executive branch is responsible for the execution and oversight of laws. It is headed by the governor of Alabama. Other members of the executive branch include the cabinet, the lieutenant governor of Alabama, the Attorney General of Alabama, the Alabama Secretary of State, the Alabama State Treasurer, and the State Auditor of Alabama. The current governor is Republican Kay Ivey.\n\nThe members of the Legislature take office immediately after the November elections. Statewide officials, such as the governor, lieutenant governor, attorney general, and other constitutional officers, take office the following January.\n\nThe judiciary is responsible for interpreting the Constitution of Alabama and applying the law in state criminal and civil cases. The state\'s highest court is the Supreme Court of Alabama. Alabama uses partisan elections to select judges. Since the 1980s judicial campaigns have become increasingly politicized. The current chief justice of the Alabama Supreme Court is Republican Tom Parker. All sitting justices on the Alabama Supreme Court are members of the Republican Party. There are two intermediate appellate courts, the Court of Civil Appeals and the Court of Criminal Appeals, and four trial courts: the circuit court (trial court of general jurisdiction), and the district, probate, and municipal courts.\n\nSome critics believe the election of judges has contributed to an exceedingly high rate of executions. Alabama has the highest per capita death penalty rate in the country. In some years, it imposes more death sentences than does Texas, a state which has a population five times larger. However, executions per capita are significantly higher in Texas. Some of its cases have been highly controversial; the U.S. Supreme Court has overturned 24 convictions in death penalty cases. It was the only state to allow judges to override jury decisions in whether or not to use a death sentence; in 10 cases judges overturned sentences of life imprisonment without parole that were voted unanimously by juries. This judicial authority was removed in April 2017.\n\nTaxes\nTaxes are collected by the Alabama Department of Revenue. Alabama levies a 2%, 4%, or5% personal income tax, depending on the amount earned and filing status. Taxpayers are allowed to deduct their federal income tax from their Alabama state tax, even if taking the standard deduction; those who itemize can also deduct FICA (the Social Security and Medicare tax).\n\nThe state\'s general sales tax rate is 4%. Sales tax rates for cities and counties are also added to purchases.</ref> For example, the total sales tax rate in Mobile County, Alabama is 10% and there is an additional restaurant tax of 1%, which means a diner in Mobile County, Alabama would pay an 11% tax on a meal.\n\nIn 2020, sales and excise taxes in Alabama accounted for 38% of all state and local revenue.\n\nOnly Alabama, Mississippi, and South Dakota tax groceries at the full state sales tax rate.\n\nThe corporate income tax rate in Alabama is 6.5%. The overall federal, state, and local tax burden in Alabama ranks the state as the second least tax-burdened state in the country.\n\nProperty taxes of .40% of assessed value per year, are the second-lowest in the U.S., after Hawaii. The current state constitution requires a voter referendum to raise property taxes.\n\nCounty and local governments\n\nAlabama has 67 counties. Each county has its own elected legislative branch, usually called the county commission. It also has limited executive authority in the county. Because of the constraints of the Alabama Constitution, which centralizes power in the state legislature, only seven counties (Jefferson, Lee, Mobile, Madison, Montgomery, Shelby, and Tuscaloosa) in the state have limited home rule. Instead, most counties in the state must lobby the Local Legislation Committee of the state legislature to get simple local policies approved, ranging from waste disposal to land use zoning.\n\nThe state legislature has retained power over local governments by refusing to pass a constitutional amendment establishing home rule for counties, as recommended by the 1973 Alabama Constitutional Commission. Legislative delegations retain certain powers over each county. United States Supreme Court decisions in Baker v. Carr (1964) required that both houses have districts established on the basis of population, and redistricted after each census, to implement the principle of "one man, one vote". Before that, each county was represented by one state senator, leading to under-representation in the state senate for more urbanized, populous counties. The rural bias of the state legislature, which had also failed to redistrict seats in the state house, affected politics well into the 20th century, failing to recognize the rise of industrial cities and urbanized areas.\n\n"The lack of home rule for counties in Alabama has resulted in the proliferation of local legislation permitting counties to do things not authorized by the state constitution. Alabama\'s constitution has been amended more than 700 times, and almost one-third of the amendments are local in nature, applying to only one county or city. A significant part of each legislative session is spent on local legislation, taking away time and attention of legislators from issues of statewide importance."\n\nAlabama is an alcoholic beverage control state, meaning the state government holds a monopoly on the sale of alcohol. The Alabama Alcoholic Beverage Control Board controls the sale and distribution of alcoholic beverages in the state. A total of 25 of the 67 counties are "dry counties" which ban the sale of alcohol, and there are many dry municipalities in counties which permit alcohol sales.\n\nPolitics\nDuring Reconstruction following the American Civil War, Alabama was occupied by federal troops of the Third Military District under General John Pope. In 1874, the political coalition of white Democrats known as the Redeemers took control of the state government from the Republicans, in part by suppressing the black vote through violence, fraud, and intimidation.\n\nAfter 1890, a coalition of White Democratic politicians passed laws to segregate and disenfranchise African American residents, a process completed in provisions of the 1901 constitution. Provisions which disenfranchised blacks resulted in excluding many poor Whites. By 1941 more Whites than Blacks had been disenfranchised: 600,000 to 520,000. The total effects were greater on the black community, as almost all its citizens were disfranchised and relegated to separate and unequal treatment under the law.\n\nFrom 1901 through the 1960s, the state did not redraw election districts as population grew and shifted within the state during urbanization and industrialization of certain areas. As counties were the basis of election districts, the result was a rural minority that dominated state politics through nearly three-quarters of the century, until a series of federal court cases required redistricting in 1972 to meet equal representation.\n\nAlabama state politics gained nationwide and international attention in the 1950s and 1960s during the civil rights movement, when whites bureaucratically, and at times violently, resisted protests for electoral and social reform. Governor George Wallace, the state\'s only four-term governor, was a controversial figure who vowed to maintain segregation. Only after passage of the federal Civil Rights Act of 1964 and Voting Rights Act of 1965 did African Americans regain the ability to exercise suffrage, among other civil rights. In many jurisdictions, they continued to be excluded from representation by at-large electoral systems, which allowed the majority of the population to dominate elections. Some changes at the county level have occurred following court challenges to establish single-member districts that enable a more diverse representation among county boards.\n\nIn 2007, the Alabama Legislature passed, and Republican governor Bob Riley signed a resolution expressing "profound regret" over slavery and its lingering impact. In a symbolic ceremony, the bill was signed in the Alabama State Capitol, which housed Congress of the Confederate States of America.\n\nIn 2010, Republicans won control of both houses of the legislature for the first time in 136 years.\n\n, there are a total of 3,589,839 registered voters, with 3,518,285 active, and the others inactive in the state.\n\nElections\n\nIn a 2020 study, Alabama was ranked as the 12th most difficult state for citizens to vote.\n\nState elections\nWith the disfranchisement of Blacks in 1901, the state became part of the "Solid South", a system in which the Democratic Party operated as effectively the only viable political party in every Southern state. For nearly a hundred years local and state elections in Alabama were decided in the Democratic Party primary, with generally only token Republican challengers running in the General Election. Since the mid- to late 20th century, however, white conservatives started shifting to the Republican Party. In Alabama, majority-white districts are now expected to regularly elect Republican candidates to federal, state and local office.\n\nMembers of the nine seats on the Supreme Court of Alabama and all ten seats on the state appellate courts are elected to office. Until 1994, no Republicans held any of the court seats. In that general election, the then-incumbent chief justice, Ernest C. Hornsby, refused to leave office after losing the election by approximately 3,000 votes to Republican Perry O. Hooper Sr. Hornsby sued Alabama and defiantly remained in office for nearly a year before finally giving up the seat after losing in court. The Democrats lost the last of the nineteen court seats in August 2011 with the resignation of the last Democrat on the bench.\n\nIn the early 21st century, Republicans hold all seven of the statewide elected executive branch offices. Republicans hold six of the eight elected seats on the Alabama State Board of Education. In 2010, Republicans took large majorities of both chambers of the state legislature, giving them control of that body for the first time in 136 years. The last remaining statewide Democrat, who served on the Alabama Public Service Commission, was defeated in 2012.\n\nOnly three Republican lieutenant governors have been elected since the end of Reconstruction, when Republicans generally represented Reconstruction government, including the newly emancipated freedmen who had gained the franchise. The three GOP lieutenant governors are Steve Windom (1999–2003), Kay Ivey (2011–2017), and Will Ainsworth (2019–present).\n\nLocal elections\nMany local offices (county commissioners, boards of education, tax assessors, tax collectors, etc.) in the state are still held by Democrats. Many rural counties have voters who are majority Democrats, resulting in local elections being decided in the Democratic primary. Similarly many metropolitan and suburban counties are majority-Republican and elections are effectively decided in the Republican Primary, although there are exceptions.\n\nAlabama\'s 67 county sheriffs are elected in partisan, at-large races, and Democrats still retain the narrow majority of those posts. The current split is 35 Democrats, 31 Republicans, and one Independent Fayette. However, most of the Democratic sheriffs preside over rural and less populated counties. The majority of Republican sheriffs have been elected in the more urban/suburban and heavily populated counties. , the state of Alabama has one female sheriff, in Morgan County, Alabama, and ten African-American sheriffs.\n\nFederal elections\nThe state\'s two U.S. senators are Republican Richard C. Shelby and Republican Tommy Tuberville. Shelby was originally elected to the Senate as a Democrat in 1986 and re-elected in 1992, but switched parties immediately following the November 1994 general election.\n\nIn the U.S. House of Representatives, the state is represented by seven members, six of whom are Republicans: (Bradley Byrne, Mike D. Rogers, Robert Aderholt, Morris J. Brooks, Martha Roby, and Gary Palmer) and one Democrat: Terri Sewell who represents the Black Belt as well as most of the predominantly black portions of Birmingham, Tuscaloosa and Montgomery.\n\nEducation\n\nPrimary and secondary education\n\nPublic primary and secondary education in Alabama is under the purview of the Alabama State Board of Education as well as local oversight by 67 county school boards and 60 city boards of education. Together, 1,496 individual schools provide education for 744,637 elementary and secondary students.\n\nPublic school funding is appropriated through the Alabama Legislature through the Education Trust Fund. In FY 2006–2007, Alabama appropriated $3,775,163,578 for primary and secondary education. That represented an increase of $444,736,387 over the previous fiscal year. In 2007, more than 82 percent of schools made adequate yearly progress (AYP) toward student proficiency under the National No Child Left Behind law, using measures determined by the state of Alabama.\n\nWhile Alabama\'s public education system has improved in recent decades, it lags behind in achievement compared to other states. According to U.S. Census data (2000), Alabama\'s high school graduation rate (75%) is the fourth lowest in the U.S. (after Kentucky, Louisiana and Mississippi). The largest educational gains were among people with some college education but without degrees.\n\nGenerally prohibited in the West at large, school corporal punishment is not unusual in Alabama, with 27,260 public school students paddled at least one time, according to government data for the 2011–2012 school year. The rate of school corporal punishment in Alabama is surpassed by only Mississippi and Arkansas.\n\nColleges and universities\n\nAlabama\'s programs of higher education include 14 four-year public universities, two-year community colleges, and 17 private, undergraduate and graduate universities. In the state are four medical schools (as of fall 2015) (University of Alabama School of Medicine, University of South Alabama and Alabama College of Osteopathic Medicine and The Edward Via College of Osteopathic Medicine—Auburn Campus), two veterinary colleges (Auburn University and Tuskegee University), a dental school (University of Alabama School of Dentistry), an optometry college (University of Alabama at Birmingham), two pharmacy schools (Auburn University and Samford University), and five law schools (University of Alabama School of Law, Birmingham School of Law, Cumberland School of Law, Miles Law School, and the Thomas Goode Jones School of Law). Public, post-secondary education in Alabama is overseen by the Alabama Commission on Higher Education and the Alabama Department of Postsecondary Education. Colleges and universities in Alabama offer degree programs from two-year associate degrees to a multitude of doctoral level programs.\n\nThe largest single campus is the University of Alabama, located in Tuscaloosa, with 37,665 enrolled for fall 2016. Troy University was the largest institution in the state in 2010, with an enrollment of 29,689 students across four Alabama campuses (Troy, Dothan, Montgomery, and Phenix City), as well as sixty learning sites in seventeen other states and eleven other countries. The oldest institutions are the public University of North Alabama in Florence and the Catholic Church-affiliated Spring Hill College in Mobile, both founded in 1830.\n\nAccreditation of academic programs is through the Southern Association of Colleges and Schools (SACS) as well as other subject-focused national and international accreditation agencies such as the Association for Biblical Higher Education (ABHE), the Council on Occupational Education (COE), and the Accrediting Council for Independent Colleges and Schools (ACICS).\n\nAccording to the 2011 U.S. News & World Report, Alabama had three universities ranked in the top 100 Public Schools in America (University of Alabama at 31, Auburn University at 36, and University of Alabama at Birmingham at 73).\n\nAccording to the 2012 U.S. News & World Report, Alabama had four tier one universities (University of Alabama, Auburn University, University of Alabama at Birmingham and University of Alabama in Huntsville).\n\nMedia\n\nMajor newspapers include Birmingham News, Mobile Press-Register, and Montgomery Advertiser.\n\nMajor television network affiliates in Alabama include:\n\n ABC\n WGWW 40.2 ABC, Anniston\n WBMA 58/WABM 68.2 ABC, Birmingham\n WDHN 18 ABC, Dothan\n WAAY 31 ABC, Huntsville\n WEAR 3 ABC Pensacola, Florida/Mobile\n WNCF 32 ABC, Montgomery\n WDBB 17.2 ABC, Tuscaloosa\n CBS\n WIAT 42 CBS, Birmingham\n WTVY 4 CBS, Dothan\n WHNT 19 CBS, Huntsville\n WKRG 5 CBS, Mobile\n WAKA 8 CBS, Selma/Montgomery\n Fox\n WBRC 6 FOX, Birmingham\n WZDX 54 FOX, Huntsville\n WALA 10 FOX, Mobile\n WCOV 20 FOX, Montgomery\n WDFX 34 FOX, Ozark/Dothan\n NBC\n WVTM 13 NBC, Birmingham\n WRGX 23 NBC, Dothan\n WAFF 48 NBC, Huntsville\n WPMI 15 NBC, Mobile\n WSFA 12 NBC, Montgomery\n PBS/Alabama Public Television\n WBIQ 10 PBS, Birmingham\n WIIQ 41 PBS, Demopolis\n WDIQ 2 PBS, Dozier\n WFIQ 36 PBS, Florence\n WHIQ 25 PBS, Huntsville\n WGIQ 43 PBS, Louisville\n WEIQ 42 PBS, Mobile\n WAIQ 26 PBS, Montgomery\n WCIQ 7 PBS, Mount Cheaha\n The CW\n WTTO 21, Homewood/Birmingham\n WTVY 4.3, Dothan\n WHDF 15, Florence/Huntsville\n WFNA 55, Gulf Shores/Mobile/Pensacola, FL\n WDBB 17, Tuscaloosa\n WBMM 22, Tuskegee/Montgomery\n\nCulture\n\nLiterature\n\nSports\n\nProfessional sports\n\nAlabama has several professional and semi-professional sports teams, including three minor league baseball teams.\n\nNotes\n\nThe Talladega Superspeedway motorsports complex hosts a series of NASCAR events. It has a seating capacity of 143,000 and is the thirteenth largest stadium in the world and sixth largest stadium in America. Also, the Barber Motorsports Park has hosted IndyCar Series and Rolex Sports Car Series races.\n\nThe ATP Birmingham was a World Championship Tennis tournament held from 1973 to 1980.\n\nAlabama has hosted several professional golf tournaments, such as the 1984 and 1990 PGA Championship at Shoal Creek, the Barbasol Championship (PGA Tour), the Mobile LPGA Tournament of Champions, Airbus LPGA Classic, and Yokohama Tire LPGA Classic (LPGA Tour), and The Tradition (Champions Tour).\n\nCollege sports\nCollege football is extremely popular in Alabama, particularly the University of Alabama Crimson Tide and Auburn University Tigers, rivals in the Southeastern Conference. Alabama averages over 100,000 fans per game and Auburn averages over 80,000—both numbers among the top twenty in the nation. Bryant–Denny Stadium is the home of the Alabama football team, and has a seating capacity of 101,821, and is the fifth largest stadium in America. Jordan-Hare Stadium is the home field of the Auburn football team and seats up to 87,451.\n\nLegion Field is home of the UAB Blazers football program and the Birmingham Bowl. It seats 71,594. Ladd–Peebles Stadium in Mobile is the home of the University of South Alabama football team, and serves as the home of the NCAA Senior Bowl, LendingTree Bowl, and Alabama-Mississippi All Star Classic; the stadium seats 40,646. In 2009, Bryant–Denny Stadium and Jordan-Hare Stadium became the homes of the Alabama High School Athletic Association state football championship games, after previously being held at Legion Field in Birmingham.\n\nTransportation\n\nAviation\n\nMajor airports with sustained operations in Alabama include Birmingham-Shuttlesworth International Airport (BHM), Huntsville International Airport (HSV), Dothan Regional Airport (DHN), Mobile Regional Airport (MOB), Montgomery Regional Airport (MGM), Northwest Alabama Regional Airport (MSL) and Northeast Alabama Regional Airport (GAD).\n\nRail\n\nFor rail transport, Amtrak schedules the Crescent, a daily passenger train, running from New York to New Orleans with station stops at Anniston, Birmingham, and Tuscaloosa.\n\nRoads\nAlabama has six major interstate routes: Interstate 65 (I-65) travels north–south roughly through the middle of the state; I-20/I-59 travel from the central west Mississippi state line to Birmingham, where I-59 continues to the north-east corner of the state and I-20 continues east towards Atlanta; I-85 originates in Montgomery and travels east-northeast to the Georgia state line, providing a main thoroughfare to Atlanta; and I-10 traverses the southernmost portion of the state, traveling from west to east through Mobile. I-22 enters the state from Mississippi and connects Birmingham with Memphis, Tennessee. In addition, there are currently five auxiliary interstate routes in the state: I-165 in Mobile, I-359 in Tuscaloosa, I-459 around Birmingham, I-565 in Decatur and Huntsville, and I-759 in Gadsden. A sixth route, I-685, will be formed when I-85 is rerouted along a new southern bypass of Montgomery. A proposed northern bypass of Birmingham will be designated as I-422. Since a direct connection from I-22 to I-422 will not be possible, I-222 has been proposed, as well.\n\nSeveral U.S. Highways also pass through the state, such as U.S. Route 11 (US-11), US-29, US-31, US-43, US-45, US-72, US-78, US-80, US-82, US-84, US-90, US-98, US-231, US-278, US-280, US-331, US-411, and US-431.\n\nThere are four toll roads in the state: Montgomery Expressway in Montgomery; Northport/Tuscaloosa Western Bypass in Tuscaloosa and Northport; Emerald Mountain Expressway in Wetumpka; and Beach Express in Orange Beach.\n\nPorts\nThe Port of Mobile, Alabama\'s only saltwater port, is a large seaport on the Gulf of Mexico with inland waterway access to the Midwest by way of the Tennessee–Tombigbee Waterway. The Port of Mobile was ranked 12th by tons of traffic in the United States during 2009. The newly expanded container terminal at the Port of Mobile was ranked as the 25th busiest for container traffic in the nation during 2011. The state\'s other ports are on rivers with access to the Gulf of Mexico.\n\nWater ports of Alabama, listed from north to south:\n\nSee also\n\n Index of Alabama-related articles\n Outline of Alabama—organized list of topics about Alabama\n\nNotes\n\nReferences\n\nFurther reading\n\n Atkins, Leah Rawls, Wayne Flynt, William Warren Rogers, and David Ward. Alabama: The History of a Deep South State (1994).\n Flynt, Wayne. Alabama in the Twentieth Century (2004).\n Owen Thomas M. History of Alabama and Dictionary of Alabama Biography (4 vols, 1921).\n Jackson, Harvey H. Inside Alabama: A Personal History of My State (2004).\n Mohl, Raymond A. "Latinization in the Heart of Dixie: Hispanics in Late-twentieth-century Alabama" Alabama Review (2002, 55(4): 243–274). \n Peirce, Neal R. The Deep South States of America: People, Politics, and Power in the Seven Deep South States (1974).\n Williams, Benjamin Buford. A Literary History of Alabama: The Nineteenth Century (1979).\n WPA Guide to Alabama (1939).\n\nExternal links\n\n \n \n Alabama State Guide, from the Library of Congress\n Your Not So Ordinary Alabama Tourist Guide\n All About Alabama, at the Alabama Department of Archives and History\n Code of Alabama 1975\n USGS real-time, geographic, and other scientific resources of Alabama \n Alabama QuickFacts from the U.S. Census Bureau\n Alabama State Fact Sheet\n \n\n \n1819 establishments in the United States\nSouthern United States\nStates and territories established in 1819\nStates of the Confederate States\nStates of the Gulf Coast of the United States\nStates of the United States\nU.S. states with multiple time zones\nContiguous United States']}

In the above code, we are loading only 10% of the training split of the Wiki dataset to make the training process faster. Feel free to adjust this percentage according to your computational resources and needs.

Step 2: Preprocess the Dataset

Before we can use this dataset for training our model, we need to preprocess the text. This usually involves cleaning the text and converting it to lowercase.

def preprocess_function(text):
    return text.strip().lower()


# Apply preprocessing to the dataset
dataset = dataset.map(lambda x: {"text": preprocess_function(x["text"])})

In the above code, we define a preprocessing function that takes in a batch of examples and returns the preprocessed text. We then apply this function to our dataset using the map method.

This concludes the preparation of our dataset. The dataset is now ready to be used for training our tokenizer and GPT-2 model.

In the next section, we will train a tokenizer from scratch using our preprocessed dataset.

Creating a Tokenizer

Tokenization is the process of splitting a sequence of text into individual tokens, which are usually words, subwords, or characters. In this section, we will create and train a tokenizer from scratch using our preprocessed Wiki dataset. We will use the tokenizers library for this.

Step 1: Initialize a Byte-Level BPE Tokenizer

GPT-2 uses a Byte-Level BPE (Byte Pair Encoding) tokenizer, so we’ll initialize one.

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# GPT-2 uses byte-level BPE
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=False)
tokenizer.decoder = decoders.ByteLevel()

Step 2: Train the Tokenizer

Next, we need to train the tokenizer on our dataset. The trainer will take care of setting the special tokens and other configurations.

# Define the special tokens
special_tokens = ["[PAD]", "[UNK]", "[CLS]", "[SEP]", "[MASK]"]

# Setup a trainer
trainer = trainers.BpeTrainer(
    vocab_size=50257,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet(),
    special_tokens=special_tokens,
)

# Train the tokenizer
tokenizer.train_from_iterator(dataset["text"], trainer=trainer)

In the above code, we first define the special tokens used by GPT-2. Then, we setup a trainer with a specified vocabulary size, initial alphabet, and special tokens. Finally, we train the tokenizer on our dataset using the train_from_iterator method.

Step 3: Save the Tokenizer

After the tokenizer is trained, it’s a good practice to save it for later use.

# Save the tokenizer
tokenizer.save("../tmp/gpt2_tokenizer.json")

from tokenizers import Tokenizer
from transformers import PreTrainedTokenizerFast

# Load the trained tokenizer
tokenizer_obj = Tokenizer.from_file("../tmp/gpt2_tokenizer.json")

# Create a PreTrainedTokenizerFast
tokenizer = PreTrainedTokenizerFast(
    tokenizer_object=tokenizer_obj,
    unk_token="[UNK]",
    sep_token="[SEP]",
    pad_token="[PAD]",
    cls_token="[CLS]",
    mask_token="[MASK]",
)
tokenizer.save_pretrained("../tmp/gpt2_uncased")

('../tmp/gpt2_uncased/tokenizer_config.json',
 '../tmp/gpt2_uncased/special_tokens_map.json',
 '../tmp/gpt2_uncased/tokenizer.json')

We have now successfully created and trained a tokenizer from scratch using our Wiki dataset. This tokenizer will be used to preprocess our dataset into a format that our GPT-2 model can understand.

In the next section, we will format our dataset for the GPT-2 model.

Formatting the Data

Now that we have our tokenizer ready, the next step is to prepare our dataset in a format suitable for training our GPT-2 model. This usually involves tokenizing the text and organizing it into sequences of a fixed length.

Step 1: Tokenize the Text

We will use our newly trained tokenizer to encode the text in our dataset. This will convert the text into sequences of tokens, where each token is replaced by its corresponding ID in the tokenizer’s vocabulary.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("../tmp/gpt2_uncased")
print(f"is_fast: {tokenizer.is_fast}")

is_fast: True

# Tokenization
def tokenize_function(examples):
    return tokenizer(examples["text"], truncation=False, padding=False)


tokenized_dataset = dataset.map(
    tokenize_function, batched=True, remove_columns=["text"]
)

Map:   3%|▎         | 21000/645867 [00:53<21:02, 494.98 examples/s]

In the above code, we define a function that tokenizes a batch of examples, and then apply this function to our dataset using the map method.

Step 2: Format the Dataset

Next, we need to organize our tokenized dataset into sequences of a fixed length. We will also shift the labels by one position so that the model can predict the next token given the current one, as required for causal language modeling.

# Formatting
block_size = 128  # or any number suitable to your context


def group_texts(examples):
    # Concatenate all 'input_ids'
    concatenated_examples = sum(examples["input_ids"], [])
    total_length = len(concatenated_examples)
    # Organize into sequences of fixed length
    sequences = [
        concatenated_examples[i : i + block_size]
        for i in range(0, total_length, block_size)
    ]
    result = {
        "input_ids": sequences,
        # Shift the labels for CLM
        "labels": [sequence[1:] + [tokenizer.eos_token_id] for sequence in sequences],
    }
    return result


tokenized_dataset = tokenized_dataset.map(
    group_texts,
    batched=True,
    batch_size=1000,  # or any number suitable to your context
)

We now have our dataset properly formatted for training our GPT-2 model.

In the next section, we will define our GPT-2 model configuration.

Defining the Model

In this section, we will define the configuration for our GPT-2 model. This involves specifying the architecture of the model and its hyperparameters.

Step 1: Import Necessary Modules

We will need to import the GPT-2 model and its configuration from the transformers library.

from transformers import GPT2LMHeadModel, GPT2Config

Step 2: Define the Configuration

We will define the configuration for our GPT-2 model. We need to specify the number of heads, the number of layers, the model size, and the size of the vocabulary among other things.

# Define configuration
config = GPT2Config(
    vocab_size=tokenizer.get_vocab_size(),
    bos_token_id=tokenizer.token_to_id("[CLS]"),
    eos_token_id=tokenizer.token_to_id("[SEP]"),
    pad_token_id=tokenizer.token_to_id("[PAD]"),
    n_positions=1024,
    n_ctx=1024,
    n_embd=768,
    n_layer=12,
    n_head=12,
)

In the above code, we have defined the configuration for a small GPT-2 model with 12 layers and 12 attention heads. The model has an embedding size of 768, and we’ve set the maximum number of positional embeddings to 1024.

Step 3: Initialize the Model

With the configuration defined, we can now initialize our GPT-2 model.

# Initialize the model
model = GPT2LMHeadModel(config)

This completes the definition of our model. In the next section, we will train this model on our prepared dataset.

Training the Model

In this section, we will train our GPT-2 model on the prepared Wiki dataset. Training a model involves defining a training loop where the model’s parameters are updated to minimize the loss on the training data.

Step 1: Define Training Arguments

We first need to define the training arguments which specify the training parameters such as the number of epochs, the batch size, and the learning rate among others.

from transformers import Trainer, TrainingArguments

# Define training arguments
training_args = TrainingArguments(
    output_dir="../tmp/gpt2",
    overwrite_output_dir=True,
    num_train_epochs=1,  # Adjust as necessary
    per_device_train_batch_size=32,  # Adjust as necessary
    save_steps=10_000,
    save_total_limit=2,
    log_dir="../tmp/gpt2_logs",
    fp16=True,
)

In the above code, we set the output directory for our model and specify that we want to overwrite any existing output in this directory. We set the number of training epochs to 1 and the training batch size to 32. These values can be adjusted based on the available computational resources. We also specify that we want to save our model every 10,000 steps, and we want to keep a maximum of 2 saved models on disk.

Step 2: Define a Trainer

We will use Hugging Face’s Trainer class to handle the training process. We need to provide it with our model, the training arguments, and our training dataset.

# Define a trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset,
)

Step 3: Train the Model

Now that we have defined our trainer, we can start the training process.

from accelerate import Accelerator

accelerator = Accelerator()
acc_state = {str(k): str(v) for k, v in accelerator.state.__dict__.items()}
device = accelerator.device

print(f"device: {device}")

trainer = accelerator.prepare(trainer)

trainer.train()

This will start the training process. The trainer will automatically handle the batching of the data, the updating of the parameters, and the computation of the loss.

In the next section, we will validate the performance of our model.

Validating the Model

In this section, we will validate the performance of our GPT-2 model. This involves using the model to generate some text and evaluating how coherent and grammatically correct the generated text is.

Step 1: Define a Generation Function

We first need to define a function that uses the model to generate some text given an input prompt.

from transformers import pipeline


def generate_text(prompt):
    generator = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
    return generator(prompt, max_length=100, do_sample=True, temperature=0.9)

In the above code, we use the pipeline function from the transformers library to create a text generation pipeline with our model and tokenizer. We set the max_length parameter to 100, which means that the generated text will not be longer than 100 tokens. The do_sample parameter is set to True, which means that the function will sample from the distribution of the next token rather than always choosing the most likely token. The temperature parameter controls the randomness of the sampling, with higher values leading to more random outputs.

Step 2: Generate and Evaluate Text

We can now use our function to generate some text and evaluate its quality.

# Generate some text
generated_text = generate_text("The history of artificial intelligence began in the")

# Print the generated text
print(generated_text[0]["generated_text"])

This will print the generated text. You should manually evaluate the text and check for coherence and grammar.

Remember that this is a qualitative evaluation and is subject to personal judgment. A more quantitative evaluation could involve computing the perplexity of the model on a held-out validation set, although this would require a labeled dataset.

In the next section, we will save and load the model.

Saving and Loading the Model

After training and validating the model, it is crucial to save the model’s weights and configuration. This allows you to load the model later without the need to retrain it.

Step 1: Save the Model

We can use the save_model method of the Trainer class to save the model.

# Save the model
trainer.save_model("../tmp/gpt2")

This will save the model’s weights and configuration to the specified directory.

Step 2: Save the Tokenizer

Remember to also save the tokenizer, as it is crucial for preprocessing the input data in the same way as during training.

# Save the tokenizer
tokenizer.save_pretrained("../tmp/gpt2")

Step 3: Load the Model

In a new session, we can load the model and the tokenizer using the from_pretrained method.

# Load the model
model = GPT2LMHeadModel.from_pretrained("../tmp/gpt2")

# Load the tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("../tmp/gpt2")

With these steps, we have successfully saved and loaded a trained GPT-2 model. You can now use this model to generate text, fine-tune it on a specific task, or even continue training it.