There is no shortage of criticism about the loosening of boundaries in diagnosing mental illness and the role of the key classification document, the DSM, or the Diagnostic and Statistical Manual. A document that arose from a crisis in the legitimacy of the profession almost four decades ago in the United States now threatens to be the very cause of a similar crisis all over again after the release of its latest edition, DSM V.

Most psychiatrists in Australia do not take the document too seriously — it is largely seen as something appropriate to satisfy insurance providers in the US. But my colleagues still use its labels when advising governments, courts or employers about disability, crime or compensation. Some include a footnote in their reports the DSM is an imperfect system that was never meant to be used for such broad purposes.

Huge rises in particular diagnoses are often related to funding arrangements, best represented by the exponential increase in the number of people diagnosed with autism. One in 160 people is now given an autism diagnosis, a 50-fold increase in three decades. Relatives of autism sufferers are also among the most vociferous defenders of the current DSM criteria.

The document’s key strength is what is known by statisticians as reliability, in that the same patient is likely to receive the same diagnosis with different providers, something that was woefully deficient before its existence. Its key weaknesses are validity and a lack of context.

Validity refers to the notion that its categories might have any basis in any biological reality. The committee that determines the diagnostic labels in psychiatry may best be described as a group of middle-aged white men who raise their hands when they agree with a certain label. In fact, most diagnoses arise after multiple rounds of horse trading, a process much closer to politics than science. The most recent reports of internet addiction being included is a case in point, for it has no basis in biology, but there are certainly cases of people losing control of their life due to online gaming.

The other key weakness is a lack of context, best described in one of the most heralded critiques of modern psychiatric diagnosis by professors Jerome Wakefield and Allan Horwitz in their 2008 book The Loss of Sadness. In it they describe how grieving reactions to significant losses, such as that of a job, divorce or bankruptcy, automatically qualify as illness. Only the death of a loved one is classified separately.

While the DSM is often accused of being a tool of professional imperialism on the part of psychiatrists, it is far more likely to be used dogmatically by non-experts — general practitioners, counsellors and members of the public.

A significant portion of my job is to convince anxious patients that they do not suffer a mental illness merely because they satisfied the checklist found on the internet. I explain that a key requirement of mental illness includes experiencing what is known as impairment, or not being able to fulfil social roles in either work or relationships for an extended period of time, at least one month. This has its roots in Freud, who said the key functions of an adult were to love and to work. Mental illness is ultimately a social definition.

Why is psychiatry forced to rely on a grab bag of symptoms for its diagnoses? Ultimately, it has nothing better to offer. In spite of the huge optimism surrounding neuroscience’s ability to explain human behaviour and, in turn, mental illness, the causes of psychopathology remain as obscure and multli-layered as ever. This is both the lure and frustration of working in mental health.

The DSM is ultimately a poor simplification of the highly complex debates that purport to explain our nature, from psychoanalysts, who insist early childhood experience interacts with with our innate instinctual drives, to biological determinists, who view mental illness through the lens of neurotransmitters, and behaviourists, who emphasise unhelpful thoughts as the originator of pathological actions.

Mental illness has become a synonym for human distress more generally, and its web can only grow as group identities based on clan, ethnicity and religion gradually attenuate and anger, hope and despair are increasingly privatised. In fact, in December 2012 an international, multi-centre trial published in the prestigious Lancet journal found that mental illness has officially replaced back pain as the world’s most common cause of disability.

In spite of its appropriate criticism, DSM and its classification of mental illness is much like democracy, as summed up by Winston Churchill. It is the worst system, except for all the others that have been tried.