Secret History: State Surveillance in New Zealand, 1900–1956

First published 2023

Auckland University Press

University of Auckland

Private Bag 92019

Auckland 1142

New Zealand

www.aucklanduniversitypress.co.nz

© Richard S. Hill and Steven Loveridge, 2023

ISBN 9781776710959

Published with the assistance of Creative New Zealand

A catalogue record for this book is available from the National Library of New Zealand

This book is copyright. Apart from fair dealing for the purpose of private study, research, criticism or review, as permitted under the Copyright Act, no part may be reproduced by any process without prior permission of the publisher. The moral rights of the authors have been asserted.

Every effort has been made to trace copyright holders and obtain permission to reproduce copyright material. The publisher apologises for any errors or omissions in this book and would be grateful if notified of any corrections that should be incorporated in future reprints or editions.

Book design by Kalee Jackson

Cover image: [The swaggering detective].

Photographs relating to waterfront dispute.

1910, 1951. R G Scott Collection.

Ref: PAColl-9508-3-27. Alexander Turnbull Library, Wellington, New Zealand

Contents

Glossary, Acronyms and Initialisms

Preface

PART ONE: CONTEXTUALISING SURVEILLANCE

Introduction: Theory and Practice

Chapter One: Surveilling Colonial New Zealand

PART TWO: PEACE AND WAR, 1900–1918

Chapter Two: Searching for Spies and Seditionaries, 1900–1914

Chapter Three: Surveillance and Suppression in Wartime, 1914–1918

PART THREE: LATENT COLD WAR, 1919–1939

Chapter Four: Searching for Subversives, 1919–1929

Chapter Five: The Red Decade? 1930–1939

PART FOUR: TOTAL WAR, 1939–1945

Chapter Six: Coercion and Dissent, 1939–1945

PART FIVE: EARLY COLD WAR, 1945–1956

Chapter Seven: Surveillance and Superpowers, 1945–1956

CONCLUDING REMARKS

Notes

Bibliography

Glossary, Acronyms and Initialisms

Agent: A person acting under direction from state security authorities to operate within targeted organisations or social circles.

ASIO: The Australian Security Intelligence Organisation, founded in 1949.

CIA: Established in 1947, the Central Intelligence Agency is the United States’ foreign intelligence service.

Comintern: Established in Moscow in 1919, the Third Communist International (Comintern) resolved to ‘struggle by all available means, including armed force, for the overthrow of the international bourgeoisie and the creation of an international Soviet republic’. It was dissolved in 1943 during the USSR’s Grand Alliance with Britain and the United States.

Counter-espionage: Efforts taken to detect, analyse and suppress foreign intelligence activities.

Counter-subversion: Efforts taken to identify, surveil, and often disrupt activities classed as posing a possible threat to the state and its interests.

CPNZ: Founded in 1921, the Communist Party of New Zealand was aligned with the Soviet Union (established 1922) throughout the period covered in this book.

Espionage: The practice of obtaining sensitive and/or classified information, typically by governments seeking information on other governments.

Front organisation: An organisation established and controlled by another organisation (in the context of this book, mostly the CPNZ) for the purpose of political advantage. In some contexts, the term is stretched to refer to instances where an organisation significantly influences another rather than holding direct control.

FSU: Founded in 1930, the Friends of the Soviet Union aimed to promote knowledge of the USSR; classified as a Communist Party front, it became defunct in 1939 after the Soviet Union signed a non-aggression pact with Nazi Germany.

GRU: See RIS.

Humint: Human intelligence is a broad term referring to traditional ways of securing information from people or organisations deemed to threaten the state or its interests, as opposed to the technical field of signals intelligence (sigint). Humint techniques range from analysing publicly available sources to clandestine methods such as infiltrating undercover agents into targeted milieus and organisations.

Informant: A person who acts as a source of information for a security agency from within an organisation, circle or movement. The nature of informants’ relationships with their handlers can be casual or ongoing, paid or unpaid, and the motives for providing information can vary (see MICE).

Intelligence cycle: A way of typologising the course of intelligence operations. Most models see the process as proceeding from the selection of targets, to the acquisition of information, and on to its analysis to produce the intelligence needed to inform further actions.

IWW: Founded in Chicago in 1905, the Industrial Workers of the World (the Wobblies) was an international labour organisation advocating syndicalist, industry-based direct action to achieve radical politico-economic change. It gained significant influence within the labour movements of a number of countries, including New Zealand.

JIB: See JIO/JIB.

JIC: The Joint Intelligence Committee. The 1946 Commonwealth Prime Ministers’ meetings agreed to establish a ‘joint intelligence system’, with each participating nation replicating the British system of a JIC which would assess intelligence requirements, review intelligence agencies and be serviced by a permanent Joint Intelligence Bureau. This system was gradually implemented country by country.

JIO/JIB: The Joint Intelligence Office was established in New Zealand in 1949 under the JIC system. A civilian assessment agency, the JIO was tasked with providing integrated geopolitical reports for the JIC, and in 1953 was renamed the Joint Intelligence Bureau in line with the other participating Commonwealth countries. Renamed and refocused several times since 1975, it became the National Assessments Bureau in 2010.

KGB: See RIS.

MI5: MI5 (officially the Security Service) is the United Kingdom’s domestic counter-intelligence and security agency, which also acquired Imperial and Commonwealth responsibilities. Its origin lies in the 1909 establishment of the Home Section of the Secret Service Bureau.

MI6: MI6 (officially the Secret Intelligence Service) is the United Kingdom’s foreign intelligence service. Its origin lies in the 1909 establishment of the Foreign Section of the Secret Service Bureau.

MICE: An acronym encapsulating the standard motives for an individual’s cooperation with intelligence authorities: money, ideology, coercion/compromise, ego.

MVD: See RIS.

NZCSO: Established in 1955, the New Zealand Combined Signals Organisation was responsible for running New Zealand’s sigint operations until 1977.

NZPF: Founded in 1886, the New Zealand Police Force’s duties included primary responsibility for security intelligence (except for a brief time during the Second World War) until 1956, when the New Zealand Security Service was established.

NZSP: Founded in 1901, the New Zealand Socialist Party was the first explicitly socialist political party in New Zealand. It reflected several strands of radical political and industrial thought and action which were rapidly emerging globally at the turn of the twentieth century.

NZSS/NZSIS: Formed in 1956, the New Zealand Security Service replaced the NZPF in holding primary responsibility for security intelligence and surveillance; it was officially renamed as the New Zealand Security Intelligence Service in 1969.

RIS: Russian Intelligence Services. A generic term used to cover a range of Soviet intelligence agencies, including the KGB (Committee for State Security), the MVD (the Ministry of Internal Affairs) and the GRU (the Main Intelligence Directorate).

SB: Established in 1949, Special Branch was the third division of the NZPF (alongside the Uniform and Detective Branches), tasked with security intelligence matters. It was wound down after the NZSS was founded in 1956 and officially dissolved in 1957.

SCR: Founded in 1941 in the aftermath of the German invasion of the USSR, the Society for Closer Relations with Russia was a successor organisation to the Friends of the Soviet Union.

Sedition: Conduct tending towards raising disaffection or discontent with, or insurrection against, the state.

SIB: Founded in 1941, the Security Intelligence Bureau (sometimes referred to as the Security Bureau) was a specialised Second World War security service which, after a damning review, came under increasing degrees of police management.

Sigint: Signals intelligence involves surveillance based around the interception and analysis of communication signals (as opposed to humint). In the context of the first half of the twentieth century, sigint essentially revolved around telegraphy, cable and wireless/radio transmissions.

Subversion: Activities deemed to challenge the structures, norms and values of an established order. A notably imprecise term, it lay undefined in New Zealand law until 1969, when it was legislated as ‘attempting, inciting, counselling, advocating, or encouraging … the overthrow by force of the Government [or] the undermining by unlawful means of the authority of the State in New Zealand’.

Surveillance: Within intelligence tradecraft this term typically refers to the specific monitoring of a target by agents on the ground or by technical devices. Within this volume the term is used, consistent with its definition by Surveillance Studies scholars, to refer to the general system of gathering information, whether through overt or covert means, on people or organisations.

UKUSA: Established after the Second World War as a top-secret agreement for signals intelligence cooperation between the United States, the United Kingdom, Australia, Canada and New Zealand, it is now commonly known as ‘the Five Eyes’.

UWM: Established in 1930 with the arrival of the Depression, branches of the Unemployed Workers’ Movement drew inspiration from the British movement of the same name and were consolidated into a national organisation in 1931.

Vetting: The assessment of a person’s suitability for access to sensitive information, taking into account such factors as loyalty to the state or vulnerability to blackmail.

Preface

This book is the first major product of a project on the New Zealand state’s surveillance of people and institutions seen as actual or potential threats to its interests. It covers the period when it was primarily the role of the New Zealand Police Force to undertake this task. Secret History will be followed by another work canvassing the years from 1956, when the New Zealand Security Service (later, the New Zealand Security Intelligence Service, or NZSIS) was established to undertake such duties, until the end of the twentieth century.

This first part of our project has proved to be a much longer exercise than we initially envisaged, essentially because of scarcity of information. What has been called the ‘secret world’ of security surveillance strives to remain secret, resisting the open inquiry that most historians have had the luxury of pursuing (including ourselves in our individual past projects). The ‘covert world is an inhospitable and obstructive research terrain’, leading to what has been identified as a ‘missing dimension’ in history.¹

A great deal of evidence has been destroyed, was never recorded in the first place, or remains classified for a number of reasons. The stated reasons focus on the need to prevent present-day enemies of the state gaining knowledge of surveillance methods, but include other factors such as protecting surveillers’ identities, even those who died many decades ago. More broadly, security agencies consider that, in the words of a top New Zealand security official, ‘even the smallest hints of how secret units operate can provide valuable information to opposing forces’.²

In view of this, it is not surprising that intelligence history remains nascent in New Zealand. The efforts of a handful of journalists, biographers, scholars and activists notwithstanding, the country lacks the detailed histories of intelligence agencies which have emerged in other countries (albeit constrained by problems relating to sources). While our attempt to fill this historiographical gap must remain ‘a report on progress’ – even more so than with most other historical endeavours – we have striven to capture the essence of state surveillance in the period covered in this work.

Our broad approach is chronological, but each chapter is internally thematised. After short introductory comments, we begin each with a contextual overview of the period covered. This is followed by a section on the state’s surveilling agencies, and the institutional and policy framework within which they operated. The next section provides extensive coverage of the modes of surveillance that were deployed, and examples of how these worked in practice. Such matters as how well the surveillers addressed the state’s objectives, and the ramifications for the type of country New Zealand professed to be, are then assessed in a separate section, followed by brief concluding remarks.

Of course, no historical research is produced within a vacuum. The emphases and findings of historical works are generally influenced by the preoccupations of the times in which they are produced. While we have our own separate personal views on the issues we canvass, we belong to the school which stresses that historians need to test their hypotheses against all available evidence, and draw conclusions based upon it; and that those conclusions should strive to be objective, however elusive a goal that may be. This approach, however difficult for an examination of state surveillance, both reflects and assists us with the main aims of this book: to provide evidence about what happened, to work out why it happened and to assess its ramifications.

Suffice to say here that we believe our findings have implications for the way New Zealanders view their past. In particular, the existence and activities of the state’s surveillers challenge some cherished beliefs about the relationship between power and public in New Zealand.

While researching this book, we were grateful for the assistance of the other two members of our original project team: Dr David Burke of the University of Cambridge Intelligence Seminar, an expert on British security intelligence history, and military historian David Filer in Wellington, who investigated aspects of signals intelligence history. We also thank Grace Millar, Redmer Yska, Aaron Fox and Denis Lenihan, who joined the project to conduct research on specific aspects of security intelligence, and the results have been published on our online Security and Surveillance History Series.

We thank, too, the numerous people who supplied us with material as they came across it. Much of this relates to our successor book, but Graeme Dunstall and Warwick Johnston, Drs Doug Munro and Jim Rolfe, and Professors Russell Campbell and Roger Boshier brought to our attention a considerable amount of material relevant to the current volume. Others provided access to information released by the NZSIS from their own personal files and/or those of family members. The most valuable information for our current purposes was kindly provided by Nick Bollinger, Beverley and Susan Price, Max Kelly, Jackie and Tina Matthews, Keith Locke, Maureen Birchfield, Gwyn Williams, Bill Rosenberg, Murray Horton, Barbara Einhorn, Clare Taylor and Nicola Saker.

Lastly, we thank those NZSIS staff who provided assistance, most particularly the Service’s archivist ‘Alex’, who facilitated the release of various documents and organised interviews with retired officers. We are indebted, too, to those former NZSIS staff who provided valuable information and insight into the operations of the secret world but who, like ‘Alex’, must remain anonymous.

We are grateful to the successive directors of the Stout Research Centre for New Zealand Studies, Professors Lydia Wevers (who, sadly, died when this book was nearing completion), Kate Hunter and Brigitte Bönisch-Brednich for their help and encouragement. We also owe similar thanks to the acting directors during a period of transition at the Centre, Associate Professor Anna Green and Professor Jim McAloon. We would like to also thank our other colleagues at the Stout Centre for their discussion and insights, especially former Research Professor James Belich, Dr Brad Patterson and Kathryn Patterson; and the Stout Centre’s administrator Debbie Levy for her support. We have discussed specific issues with various visiting scholars who have passed through the Centre since we began this research, especially former J. D. Stout Fellows Sarah Gaitanos and Nick Bollinger, and we thank them for their interest and assistance. Of course, errors, omissions and interpretations are entirely our own responsibility.

We are grateful to our families and friends for their interest and assistance. Richard Hill is indebted to his wife, Nicola Gilmour, for her loving support during this project – and getting him through the dark days of health difficulties in the middle of it. He has appreciated the support and interest of the circle which meets on Friday evenings in a Wellington pub (in recent years the Hotel Bristol, which happens to feature in a footnote in this book) to discuss many issues, some of them academic. Steven Loveridge is grateful for the patience extended by friends and family members during an investigation into a history murkier than any of his previous projects.

We owe especial gratitude to Drs Bob Tristram and Nicola Gilmour, David Filer and our publishers’ anonymous readers, all of whom who gave us valuable feedback on our draft manuscript. We would especially like to thank Auckland University Press: its director Sam Elworthy (who has been supportive of the project since its inception), our editor Matt Turner and proofreader Mike Wagg, designer Kalee Jackson, the press’s production manager Katharina Bauer, production assistant Lauren Donald and publishing associates Sophia Broom and Kapua O’Connor. Finally, we owe our deep gratitude to the Marsden Fund of the Royal Society of New Zealand for its generous funding support for the project.

Richard S. Hill and Steven Loveridge

Stout Research Centre for New Zealand Studies

Victoria University of Wellington/Te Herenga Waka

April 2022

PART ONE:

CONTEXTUALISING SURVEILLANCE

Introduction: Theory and Practice

Preamble

This investigation of state surveillance between 1900 and the mid-1950s seeks to cast light on a largely hidden dimension of New Zealand history, that of the ‘secret world’ of security intelligence.¹ During this period the country was generally depicted, at home and abroad, formally and informally, as a small country (with fewer than a million people in 1900 and some 2.2 million at the end of our period) blessed with an exceptionally ‘free and open’ society – a reputation that had existed since its earliest days as a colony.

Over this same period, the state possessed and employed standard processes and mechanisms for surveilling people, activities and organisations considered to pose a real or potential danger to the political order and the socio-economic system over which it presided. This imperative reflected an ancient pattern of statecraft – as examined by theoreticians as disparate as Kautilya, Sun Tzu, Niccolò Machiavelli, Carl von Clausewitz, Thomas Hobbes and Mikhail Bakunin – of rulers keeping watch for social and political elements suspected of harbouring intentions to damage, undermine or overthrow their authority.² Pursuing such suspicions always means that the surveilling net is cast far beyond those known to be actual enemies of the state and its interests to encompass much larger numbers seen as having the potential to be so.³

In the New Zealand of our period, this function of political surveillance fell, in most part, to specialised personnel within the Police Force tasked with uncovering and monitoring subversion or espionage (although we stress that the covert surveillers also drew on routine uniformed police and detective observations, material from other agencies and publicly available information).⁴ Deploying men sworn into the ancient office of constable to carry out political surveillance made sense. In the final analysis, what makes constables different from other state servants is their power to coerce – to arrest and prosecute. To secure information to do so, police personnel are empowered, on a 24/7 basis, to monitor people and places and to collect material about them. In this sense, all police are surveillers.

Our concern, however, goes beyond this general surveilling of society. It focuses upon those men in the detective wing of policing who were tasked with surveilling the political landscape. They formed a subset of policing which we term the ‘political police’, although this was not a contemporary term in New Zealand. These detectives worked within a pre-existing society-wide monitoring apparatus which fed information into the political surveillance system. Eventually they established a database of ‘secret files’ on people and the causes they were associated with.⁵

These files were generally inaccessible to other police, and the people recorded in them were typically unaware of such monitoring, although some suspected. Those few who did find out had virtually no chance of assessing the accuracy of any reports, knowing the uses to which they were put, or finding out that they centred on political beliefs and activities rather than breaches of the law. The material that built up in such files, in other words, was of a profoundly different nature to that held on non-political suspects. Political targeting sought to contextualise the whole person, gathering up information on attendances at protests, habits and beliefs, family and friends, living and working arrangements, social and political circles, and much more besides. The files on people were increasingly cross-referenced to those on subjects (organisations, movements, activities, causes) and other targeted individuals.

The existence of such extensive surveilling of those deemed politically suspect might seem puzzling in a country such as New Zealand. From the beginning, both its colonising entrepreneurs and its officials had envisaged and portrayed it as a ‘Better Britain’ of the South Seas, an antipodean paradise of abundant natural resources whose subjects could thrive in an atmosphere of freedom, unfettered by the constraints of the old country.⁶ In this supposedly ideal society, the means to ‘get on’ and improve one’s lot were promised to the settlers (and Māori, if they assimilated). These opportunities were said to be based around such attributes as fairness and egalitarianism (even classlessness), social and legal justice, freedom from want and control, and (before long) the gradual introduction of the right to vote within a parliamentary democracy.⁷ New Zealand, then, would be an exceptional country, an attribute often summed up in such catch cries as ‘a fair go for all’, ‘God’s Own Country’, ‘a free land’ and (to use a term that came into currency through its adoption by Karl Popper, who wrote his classic work on the subject in New Zealand) ‘an open society’.⁸

Such visions became firmly entrenched within New Zealand’s ‘mythscape’. That term signifies the arena in which the totality of a nation’s overarching stories and ideals ‘are forged, transmitted, negotiated, and reconstructed’ through time. These ‘myths’ – ‘highly simplified narratives [that] help constitute or bolster particular visions of self, society and world’ – might be rooted in either actual or imagined circumstances.⁹ Such subscription to exceptionalist notions, in both official rhetoric and in popular belief, cannot be dismissed as entirely chimerical. It reflects in part the attainment of real economic, political, social and other aspirations among large sectors of the population. The most recent economic history of the country places New Zealand’s Pākehā (European) settler population in 1900, in aggregate, as ‘probably among the best-off people in the world’.¹⁰ The country’s attainments, often state-assisted, gave credence to official rhetoric on such matters of representative democracy, education, and measures to provide greater social security.

The gradual construction of the mythscape, however, did not reflect the socioeconomic inequalities inherent to the type of society which was imported. There could not be, in particular, a ‘classless society’, because the political economy was based on the concept of class and therefore inequality. In 1872 Marxist activist James McPherson laid out the case that burdens and benefits were grossly misallocated, to the worker’s cost: ‘is there any country in the world that has to keep up at the highest rate so many [capitalist] drones?’¹¹

Other fractures in the national narrative can also clearly be seen. A later trope, that New Zealand possessed ‘the best race relations in the world’, sat in stark contrast to the dispossession and marginalisation of Māori. McPherson himself leavened his call for social justice with another demand, one that resonates across a wider history in New Zealand: that ‘vast hordes of Chinese’ immigrants should not be permitted to descend ‘like locusts’ and ‘eat up the just fruits’ won by European workers.¹² Moreover, while female enfranchisement in 1893 boosted the country’s international fame as a ‘social laboratory’ towards the end of the nineteenth century, gendered constraints familiar to other times and places did not disappear with the introduction of the vote.

Governing myths are always challenged by sections of the population. There were essentially two broad and often overlapping ways of contesting them: challenging the validity of some or all of the dominant elements of the mythscape, and demanding that the authorities take steps to turn proclaimed myths into reality. The secret world has a distinctive association with both the mythscape and the public. It is, by definition, a detached relationship: ‘secrecy’ derives from the Latin secretus, meaning set apart. We contend that adding the history of the secret world and its methods to the study of New Zealand’s past presents a new perspective on the complicated and blurred boundaries between myth and reality. Most significantly, as the following chapters and the succeeding volume stress, state surveillance is part of the history of New Zealand’s state and society, and a significant one at that. In regard to practising this ancient statecraft, at least, New Zealand is not exceptional.

Intelligence as Statecraft

The essence of the secret world of political policing lies in the monitoring of perceived or actual threats to state power, the interests which state power protects and the social norms it endorses or promotes (the ‘political economy’). In the nineteenth century, the Russian revolutionary anarchist Mikhail Bakunin submitted that:

Forbidden fruit has such an attraction for men, and the demon of revolt, that eternal enemy of the State, awakens so easily, in their hearts when they are not sufficiently stupefied, that neither this education nor this instruction, nor even the censorship, sufficiently guarantee the tranquillity of the State. It must still have a police, devoted agents who watch over and direct, secretly and unobtrusively, the current of the peoples’ opinions and passions.¹³

The forms such surveillance has taken, however, have varied through time and across place and regimes. Many surveilling jurisdictions around the globe, past and present, possess the authority to detain people and to initiate or administer discipline to them.¹⁴ Some jurisdictions explicitly criminalise people for their thoughts; a New Zealand teacher in interwar Japan, for example, was charged with being a ‘thought criminal’.¹⁵ While assigned members of the New Zealand Police Force (NZPF) were the major political surveillers in our period, with the ability to arrest and prosecute, their work was a far cry from that of the security police of totalitarian, authoritarian or other regimes whose powers or practices have extended to torture or killing.¹⁶ These are generally called ‘secret police’, and it is partly for this reason that we have avoided that term to describe New Zealand’s secret world.¹⁷

Our use of the term ‘political policing’ is a reminder that, in the broadest sense, all state policing functions to enforce the will of a society’s governing bodies, as expressed through legislation, executive orders and guidelines, judicial rulings and the like; and that any threat to the state or its interests is inherently political. More specifically, however, our usage highlights police surveillance of those suspected of challenging peace and ‘good order’ – that far-from-neutral term which, in the final analysis, amounts to the desired or current configuration of the political economy.

In all times, those tasked with carrying out surveillance approach their duties within wider social and institutional contexts. Each chapter in this work seeks to ground surveilling policies and operations within these broader trends, reflecting developments in New Zealand and abroad. Our chronological grounding of chapters recognises the shifting context in which the surveillance agenda was formed across the first half of the twentieth century and, in turn, impresses the place of the secret world within the events of general history. The decades reviewed contain seminal developments: the volatile political realignments which played out towards 1914; the war regulations wielded during the First World War; the impact of interwar concerns about a Moscow-sponsored threat to the country (which we conceptualise as the ‘latent Cold War’); the profound impacts of the Second World War; and the onset of the early Cold War from the later 1940s.

Examining the history sequentially also allows for appreciation of the evolution of the country’s secret world: gradual systematisation and specialisation, sudden or longer-term responses to stimuli, ad hoc or planned developments, new institutional and policy bases, implementation of legislation. The degree of state surveillance increased enormously between 1900 and 1956, a factor reflected in the lengthening of our chapters as they move through time.¹⁸

We have curated our subject by various means. In particular, we have focused on the state’s own protectors (while also covering its use of non-state actors to assist it), the policies and rules under which they operate, and the operations they undertake. While definitions of the state are heavily contested, policing and surveillance scholars often utilise one formulated by Max Weber. That is, the state is a polity that has successfully claimed ‘the monopoly of the legitimate use of physical force within a given territory’.¹⁹

Such a definition, and variations thereof, reflect the exercise of sovereignty, which might best be defined in the final analysis as ‘the power to make and enforce law’. This right was claimed under British and international law when New Zealand was annexed in 1840. It would soon be challenged by Māori tribes, which did not see their arrangements with the Crown in such light, but the state’s nominal acquisition of sovereignty gradually became substantive as resistance was quelled.²⁰

Within such a broad, sovereigntist definition, our use of the term ‘the state’ tends to be mechanism-based: namely, the totality of the various legally constituted agencies of governance and control. We pay particular attention to the political executive at the apex of the state, and those institutions which implement its surveillance policies, as well as the policies emanating from other decision-making layers of state, including the legislature and judiciary. Our key focus is placed on what is generally called human intelligence (humint): the gathering of information from human sources, by means ranging from information acquired from publicly available sources to the use of covert agents. This ‘on the ground’ work, collecting data on those who generally do not know they are the subjects of surveillance, is usually contrasted with the technical field of signals intelligence (sigint). This is concerned with the protection, interception and decryption of signals sent above the ground or below the sea. In the period covered in this book, sigint generally entailed interception and analysis of messages sent by wireless transmission.²¹

We emphasise that humint and sigint are not mutually exclusive, and that an intelligence-gathering enterprise is ‘not an end in itself’: it ‘supplements other measures [and] seeks to fill in their gaps and extend their reach’.²² Accordingly, humint practitioners will call upon sigint expertise, and (to a lesser degree) vice versa, as well as interact with many other state and private players. More broadly, the trajectories of the two broad modes of surveillance are similar, reflecting the evolving concerns of New Zealand’s decision-makers on issues of state security, both internal and geopolitical.

This is an appropriate point at which to note that this study utilises a wider definition of ‘surveillance’ than that employed by modern intelligence practitioners. Security officials and their agents in the field tend to use the word to describe the practice of physically tracking or otherwise observing a target, an alternative term to ‘watching’ (surveillers in Britain’s domestic Security Service/MI5 are internally referred to as ‘watchers’). Our use of surveillance, however, reflects the broader definition used within the international arena of surveillance studies: scrutiny, generally covert, of persons, institutions and activities believed to present a political threat to the state and its interests.²³

This definition expands surveillance to include the processes by which targets for monitoring are selected, the means by which information on them is gathered, and the ways in which this material is recorded, analysed, acted upon and archived. In other words, our interest goes well beyond the clandestine surveillers at the coalface to encompass the systems of state surveillance, the reasons for them, and their implications. The modes of surveillance we outline often involve the practitioners’ definition of surveillance – on foot, by car or from observation post – but it will also include anything from mundane office duties, such as clipping newspaper articles, to activities more popularly associated with spying, such as bugging rooms or tapping telephones.

The military tended to be the major (if intermittent) challenger to the political executive’s allocation of the humint function to the Police during our period, and it rapidly expanded its domestic role during wartime. Whatever their aspirations, however, in peacetime the defence institutions were relegated to specialist roles in areas such as sigint (as well as, of course, monitoring internal matters inside the armed forces). They are therefore not given prominence in this book, except where they are assisting the Police Force, although as with sigint per se their activities are also drawn upon to provide context. This rule of thumb also applies to other state agencies with specialised surveilling functions, such as those dealing with customs, censorship, and post and telegraph functions.

All of these serve as reminders that, while our focus is necessarily upon the humint activities of the NZPF, security intelligence was a holistic, state-wide concern of the political executive. Its surveilling gaze, moreover, was shaped by New Zealand’s position as an isolated outlier of the British Empire and Commonwealth and, later, of the American-dominated alliance which had been firmly cemented into place by 1956. New Zealand’s domestic security was always tightly intertwined with geopolitical considerations.

Producing Intelligence

At any given time, political surveillance reflects the state’s contemporary assessment of the nature and scope of threats to itself and the ideologically based order it reflects and protects. Across time and place it seeks to locate and track people and organisations which it sees as subverting that order or being capable of doing so under the influence of alternative ideologies or their representatives. The way in which the state’s perceptions of threat are operationalised can be envisaged as an ‘intelligence cycle’, a concept used to capture the fundamental processes by which the intelligence community produces its end products. This cycle forms a specialist subset of the state’s broader surveillance of social and political dynamics, an enterprise itself generally dominated by the normal mechanisms of policing.

While there are some variations in envisaging the intelligence cycle, it conventionally begins when surveillance agencies seek out and select targets – individuals, movements, organisations – for particular attention. In the second phase of the cycle, the surveillers proceed to collect information on their targets through various means, including clandestine observation. During the third phase of the cycle the raw material that has been gathered is subjected to analysis, often at increasing levels of seniority, in a quest to assess its credibility and discover patterns and implications. In some assessments, ‘it is at the analysis stage that information is converted into intelligence’.²⁴ If very firm conclusions about the targets can be reached, the analysis phase will produce ‘finished intelligence’.

The fourth and final stage of the intelligence cycle involves the use of analysis by relevant authorities (within intelligence agencies or outside them, including politicians) to inform actions on countering the threats to the state which have been identified: the surveilling gaze, in other words, provides the foundation for exercising interventionary police powers. The outcomes sought from targeted surveillance, analysis and action might be short term, dealing with the current needs of state security (such as an operation to neutralise a perceived immediate threat), or longer term – especially the accumulation of material and assessments to provide a base for any necessary future investigations and interventions.

This is, of course, a schematised depiction of the secret world, whose processes both overlap and are generally far more opaque than in most bureaucracies. In the event of a perceived national emergency, for example, operational decisions may be taken on the basis of preliminary and undigested information. At the other end of the scale, investigations of suspected agents of foreign powers may take place over decades, with outcomes produced along the way – or not. Given such messiness of process within the actual secret world, the stages of the cycle may be both differently formulated and also sub-categorised in a number of ways. A former senior British intelligence practitioner, for example, posits ‘four types of information that can form an intelligence product’: situational awareness, explanation of what is observed ‘and the motivations of those involved’, assessment of ‘how events may unfold under different assumptions’ and ‘[s]trategic notice of future issues’.²⁵

Whatever the schematic framework, we stress that all of them reflect dynamic and often unpredictable processes. The prioritising of targets often needed to be reassessed, for example, for a number of reasons – receipt of new information, routine review, fall-off in activity, political crisis, resurgence of dissidence in society, and so on. In its application to a targeted person, moreover, the intelligence cycle will endemically intersect with a large amount of other targeting and tracking activity. Keeping watch on a target, in fact, will generally lead to others within their political and social networks, some or all of whom, if they are not in it already, will be seen as eligible for entering the first phase of the intelligence cycle. When that happens, the cycle begins anew.²⁶

The second step of the cycle – more-systematic collection of material on the targeted individual, organisation or movement – differs in intensity, scope and outcome according to many variables. Its overlap and interaction with the third, analytical phase of the cycle, moreover, is often especially important. Analysis might begin, for example, as preliminary comments in an operative’s surveillance report, and end in conclusions reached during a meeting of senior police gathered in the Commissioner’s office, or sometimes at a top-tier meeting in the presiding minister’s office. This might lead to a variety of outcomes at the fourth stage of the cycle: a decision to intensify surveillance over the target(s) and their circles, for example, or even to mount a significant intervention, such as a raid on activists’ premises or arrests.

We contextualise our investigation of the intelligence cycle in terms of three interrelated sets of enduring dynamics. First, the political and socio-economic system within which the cycle operates, and which it serves. Second, the fundamental nature of the institutions that select the targets, gather information on them, analyse it and (on behalf of or in conjunction with the political executive) decide upon its use. Third, the institutional powers and objectives of the surveillers, which both reflect and shape the essential nature of security intelligence.

The first phase of the intelligence cycle, the search for targets, reflects an inherent proactivity by political surveillers. This aims to detect and evaluate the seriousness of potential threats, as opposed to reacting once threats have been mobilised – although security officials also, of course, need to respond quickly to unforeseen ‘threat activities’. Even uniformed members of the NZPF, in the period under study, frequently initiated the gathering of the information by which targets were selected. They routinely attended public meetings, marches and demonstrations to ensure public order was maintained, but also passed on to detectives any concerns they might have on grounds of security, broadly defined. If they were not already doing so, detectives, especially those specialising in surveilling the political threatscape, would attend future such gatherings, or arrange for uniformed men to don plain clothes, attend and report.

Plain-clothes policemen in attendance at such events would be searching for future targets as well as securing information on the already targeted. Their means might include asking questions of others or tailing suspects, by foot or (later) car, when they left. They might later follow up what they have observed or learnt, perhaps by intercepting mail or other communications. Such clandestine techniques, foundational to surveillance statecraft, remained fairly constant throughout our period. While they were increasingly assisted by improvements in technology, such as the ability to access phone conversations or listen in to private meetings, these represented changes in degree rather than in kind. The exception, perhaps, is radio communication which, as the following chapters track, developed from an experimental science to a readily available technology which brought new security implications and opportunities. Indeed, we postulate that a New Zealand detective involved in political policing on the first day of 1900 would, with a minimal adjustment for such matters as improvements and innovations in transport and communications, be able to readily comprehend the rhythms of political policing on the last day of 1956.

At the end of our period, as at the beginning, information gatherers might be drawn to a target through ‘open-source’ information, such as speeches at public events, newspaper articles, ephemeral or substantial literature produced by targeted individuals or organisations or, later, radio broadcasts. Any investigation typically included a painstaking extraction of material from such sources, which constitute a significant percentage of the material lodged in those New Zealand security files which have been released. In the United Kingdom, it has been estimated, ‘75 per cent of all information monitored by Special Branch officers is either publicly available or freely given’ by members of the public.²⁷

When it was deemed necessary or expedient, open-source information led to covert gathering. This was mostly collected by operatives (in our period, usually members of the NZPF), or by their agents or informants reporting from within targeted organisations or milieus.²⁸ Police surveillance techniques typically ranged from covert attendance of political gatherings to such clandestine methods as accessing phone conversations, listening in on private meetings, talking to neighbours or relatives of targets,²⁹ intercepting mail or other communications and illegally entering premises.³⁰

As the twentieth century progressed, the overall surveilling trajectory was that of greater intrusiveness into civil society. We adapt the police concept of a dragnet to describe this phenomenon. The policing dragnet is generally characterised as a close search of targeted areas and groupings to locate suspected offenders against the criminal code. In political policing, the net is not only a permanent fixture but it also hauls in the names and details of large numbers of people and groups who are usually far from being criminals. These are metaphorically netted into hidden files because their causes and actions indicate, in police eyes, that they are either threatening the state or have the capacity to do so given a certain conjuncture of events – widespread civil discontent, for example. In times of civil or wartime emergency, in particular, the dragnet will hugely extend its reach and depth, leading to even greater swelling of the secret files.

Increasingly, the types of information gathered went considerably beyond the narrowly political. As well as their basic personal details (physical description, occupation, address, driver’s licence, relatives and friends, car ownership and so on), and their political affiliations and beliefs, data gathered and filed on individuals often came to include information or speculation about mental and physical health, alcohol consumption, charisma (or lack thereof), financial circumstances, marital discord or unfaithfulness, personal relations, sexual proclivities, and so on.

This trend towards catch-all information gathering in the global history of intelligence sought to construct a holistic picture of a person, organisation or movement. Each new item of information, whatever its reliability or even its apparent inconsequence, is collected to enhance the database from which analysis is conducted and hence the chances of successful outcomes. Classically, those looking for threats to the state and/or the national interest watch for four key characteristics. These can be encapsulated in the acronym MICE: money, ideology, compromise/coercion (especially blackmail) and ego.³¹

While ‘ideology’ (broadly defined to include values, beliefs, causes and so on) predominated as the grounds for identifying targets in the first place, information relevant to the other three identifiers was also sought in order to assess the degree of danger targets posed (and, sometimes, the opportunities their susceptibility factors presented for ‘turning’ them into working for the surveillers). The lifestyle and personality of a target might, for example, be seen to render that person amenable to bribery, blackmail or flattery by domestic subversives or a foreign power.

The police mandate to keep ‘peace and good order’ essentially meant that any perceived challenge to broad social, economic and constitutional arrangements could be seen as justifying further scrutiny and potentially covert surveillance – were any justification required. On behalf of the political executive, then, humint practitioners conflated the interests of the existing political economy (and the beliefs which sustained it) with those of the state and society as a whole. Any threat to a part of the order of things could be perceived as a threat to its entirety. Thus, while very few targets wanted to fully overturn the institutions or legitimacy of the state and all that it stood for, advocates of many types of political or social reform were surveilled in the name of the ‘public good’.

Another way of putting this is that the secret world was in essence a conservative one, following the etymology of conservatism, whose goal is to conserve what is established. Conservatism reacts adversely to any perceived threats to the existing order and is generally slow to alter its views or methods as social perspectives change, although it does adapt as long as the changes are not foundational.³² The conservative disposition had ramifications for all stages of the intelligence cycle, but most especially the third phase, that of the formal analysis of collected information. Again, we note the overlap between phases in the cycle, with analysis (for example) often beginning in a preliminary way at the point of selecting targets and gathering raw data on them. We also stress that the formal stage of analysis – of collating and interpreting information of disparate origin and nature – is mostly undertaken at a higher level than that of the collectors. The task of drawing out the larger significance and meaning of often fragmentary and decontextualised material, in fact, is generally acknowledged to be ‘the hardest and most unforgiving task in the intelligence cycle’. In particular, the practitioners are aware that their analyses hold potential consequences of great import for the state (not to mention the persons surveilled).³³

Naturally, such assessments reflected the conservative paradigm within which the surveillers worked, with any challenges to the existing political, social and economic order considered to be threats to state security. Thus, people calling for reforms that ultimately became part of the public good through legislative enactment, or organisations with no overtly political stance, could be categorised as potential, if not actual, threats to the state and its interests. Fabian socialists at the turn of the century who wanted greater public input into local government, or members of the Left Book Club in later decades, were targets for investigation; as were promoters of birth control or a universal language, or advocates of various types of social security, including the ‘cradle to the grave’ welfare reforms cemented into legislation in the later 1930s.

Even if they had supported only a single-issue cause, and had not been assessed as subversive, people caught in the surveilling dragnet remained in the files, part of a pool of potential ‘dupes’ who might be manipulated or radicalised by enemies of the state.³⁴ A 1976 review of New Zealand’s Security Intelligence Service observed that surveillance authorities operated on the ‘undoubtably correct’ basis that ‘subversive activity may develop from a movement not originally subversive in concept’, giving counter-subversion work ‘broader dimensions’ around the prioritised ‘inner core’. It went on to note that ‘[m]ovements which are, or may be, the target for penetration by subversive groups or individuals are studied. So too are activities which might become subversive, including those of groups engaged in militant or anti-establishment protest or radical dissent.’³⁵ This approach represented continuity with that of the police security authorities during the period covered by this book.

The collection of information and the production of intelligence often led to no operational outcome for the state. Indeed, a recurring theme in the accounts of disenchanted former members of various intelligence agencies is the sheer accumulation of reports considered to be ‘an enormous waste of time and money’.³⁶ That being said, as we demonstrate across the following chapters, intelligence was always utilised in a variety of ways to inform the decisions and actions of state actors. Soviet and American intelligence services, respectively, used the terms ‘active measures’ and ‘covert actions’ for their efforts to influence events, distinguishing these from passive study of targeted persons and organisations. Such measures extended to arranging disinformation campaigns, sabotage and assassinations. While intelligence interventions in New Zealand were far less extreme, they did use such proactive measures as suppressing activist or alternative literature, leaking material to journalists, and intimidating people and organisations in various ways.

The most common use of gathered intelligence in this period was to inform policing measures and further surveillance operations. Thus, political police studied targeted movements and might employ various policing powers or regulations to hamper them. Some of these measures were of a political nature, such as those targeting ‘seditious utterance’, but such charges were generally lightly used as they were hard to prove to judicial standards. Often, instead, public-order offences would be invoked, such as ‘breaching the peace’, having no ‘lawful means of support’, using ‘offensive language’, being ‘idle and disorderly’, ‘obstructing a carriageway’ and so on. The monitoring of beliefs and activities seen as possible threats to the state’s integrity or existence was, therefore, treated as a subset of the general enforcement of peace and good order. The effect of this jurisdictional permeability within the policing function was that people under surveillance on political grounds were placed in the same broad category as offenders against the criminal code.

In the early years of the century, lack of operational follow-through was sometimes the result of limited resources, with only small numbers of political detectives and others available for intervention, such as working up a prosecution. The widening of the surveillance dragnet, and the casting of the surveilling gaze upon the increasing numbers it netted, prompted the development of criteria for prioritising detective actions according to the nature of the perceived threatscape. Thus, the most urgent priority for attention during socio-political turmoil might involve raiding and arresting, while a longer-term threat perception might lead to a decision to infiltrate a movement or to intensify surveillance of an organisation or persons. Most analysis, as in all intelligence services, did not lead to any immediate operationalisation.

The dragnet approach changed little in our period, other than extending the net across the proponents of new causes and, if gradually and reluctantly, ceasing intrusive surveillance of those whose ideas had become widely accepted in society and/or government policy. Along the way, however, the surveilling gaze had been assisted by technological advances; these not only enabled the widening and deepening of the surveillance net but, relatedly, also helped increase cooperation with overseas agencies. In sigint especially, but also in humint, New Zealand’s intelligence links with Australia and Britain expanded over our period. After the Second World War these three countries, together with Canada, came to form an Anglo security and intelligence bloc within the five-nation security alliance that lasts to this day. The fifth partner, the United States, had become dominant, especially in the field of sigint, by the mid-1950s.

At home, the political police had effective autonomy of operation in the various stages of the intelligence cycle, a subset of the more general operational autonomy which the NZPF had forged from their political masters since the later nineteenth century. This trend was assisted by the reality of the world of mainstream politics. Even the Labour Cabinet that formed in 1935, dominated as it was by former surveillees, soon let its humint surveillers get on with the job virtually unimpeded. But few operational decisions within the secret world involved shutting down scrutiny of a targeted cause or organisation, let alone erasure of the information held in the files long after this could have had, on the most generous assessment, any operational ramifications.

Those who had been caught up in the surveillance net, then, generally never really escaped it, whatever their later personal and political trajectories. Whether or not they knew it, and they seldom did, the files generally remained in existence throughout their lifetimes, and markings on them indicate continued referencing by security operatives.³⁷ Information in this secret archive, including much material of uncertain veracity or faulty analysis, could be drawn upon for formal vetting for Public Service positions or informal warnings to private-sector employers about prospective or current employees. Those whose careers suffered, sometimes long after the initial targeting, often did not know why they had been rejected or ejected.

Such results were but one of many consequences of proactive policing work that tapped into the files created by the surveilling net and interpreted them within the conservative paradigm of the secret world. As we have noted, New Zealand’s surveillers used methodology of ancient lineage, methods which were inherent in surveillance tradecraft worldwide. It is also equally clear that, by 1956, the widening and deepening of the surveillance dragnet meant that ‘mission creep’ had occurred over the decades; that state penetration of civil society had advanced well beyond the remit of the political policing of 1900.³⁸

Operating Environment

The relationships between political surveillers, the public and the government, always complicated, are all the more so when the secret world lies within a police force. The institutional environment of the political police, then, is central to our exploration. Major issues raised include the degree to which the political executive is able, or seeks, to balance the surveillers’ accountability to itself (on behalf of Parliament and, ultimately, the nominal head of state) with the operational autonomy sought by the surveilling institution. This is potentially a more delicate and fraught subject for the secret world, and for the politicians officially in charge of it, than for most areas of government.

Insufficient accountability to the political executive (and, more broadly, Parliament) on the part of the surveillers can lead them into operating virtually as a state within a state. The consequences might adversely affect the reputation of the state and the rights of its citizens alike: the use of dubious or illegal methods, intrusive monitoring on political or ideological grounds that fall far short of any reasonable definition of subversion, damage to the public trust and individuals’ lives, and the like. Objectively this means, from the state’s point of view, inefficiencies in terms of its goals. Insufficient autonomy for political police, on the other hand, can lead to (or at least give the impression of) party-political and other untoward interference by ministers and Members of Parliament. As we have noted, state-led surveilling mechanisms and operations are, by definition, political. In a liberal democracy, however, they are not supposed to reflect partisan interests (although this issue is complicated by the fact that the government’s policies and legislation will most likely be seen as partisan by its opponents).

One study posits the appropriate balance between accountability and autonomy as a ‘grand bargain’ by which ‘policymakers will not ask or force