Brian Harrison's Indexing System

HOW MY DATABASE CAME INTO EXISTENCE
BRIAN HARRISON 6 April 2021


I learned to type as a teenager because my father had an old typewriter and let me use it. So I took notes on books for A-level, but didn’t touch-type. When I came out of the army I became very conscious that despite my long education I had no skills that would make me employable, and so decided to teach myself to touch-type, and did so from one of those yellow-covered ‘Teach Yourself’ books, and very painful it was. I recall as an undergraduate counting 109 mistakes on a single page, and almost gave up, but fortunately didn’t, and was rather unusual as an undergraduate in typing out my tutorial essays. By the time I became a graduate student I’d become quite a fast touch-typist.
I was quite oriented towards card-indexes as an undergraduate but can’t now recall why. All I recall is that I had lots of quotations on cards that I learned by heart for Modern History Schools, and also lots of dates, also learned by heart. When I began on graduate work, I went to James Joll (Senior Tutor at St.Antony’s), assuming that he would be able to tell me how he thought one should set about organizing one’s research material, and his reply was simply to say that he could not abide card-indexes because they tended to fall out of their containers all over the floor. I didn’t then know about Keith Thomas’s ‘envelope’ methods, so in this respect wasn’t influenced by him, though in a rather different way in his own research procedures he was doing exactly what I was doing. As far as I recall, I simply developed a system for myself in 1961: typed-up notes on numbered foolscap sheets consecutively as I accumulated them and placed in ring-files, cross-indexed on small cards the size of visiting cards, which eventually in paper format I had custom-made for me either at Hunt’s in the Broad or at Emberlin’s on the corner of the Turl and Market Street (I can’t now recall which). There are now 80 of these ring-files shelved behind me as I write, numbered from 1 to 29,978. I typed into the files anything that I thought would be of research value, and the proportion of newspaper cuttings pasted on to the sheets increased as my research interests moved on from the Victorian period towards contemporary history. Initially, the notes and card-index were entirely concerned with the sources I used for writing Drink and the Victorians, The latter opened out in so many directions that I recall thinking to myself, once it seemed likely that I’d be researching in history for the rest of my life, that I must adapt the system which I’d devised simply for the purpose of qualifying for a D.Phil. into a system that would be the basis for all my subsequent research and teaching. This simply entailed developing new sections and sub-sections in the card-index to accommodate wider subjects, e.g. ‘towns’, ‘industry’, ‘religion’, ‘Chartism’, party politics and so on. So the two drawers of cards concerned with drink and temperance grew into a system concerned with the whole of modern British history, and that is the system that is now stored in our garage undigitized, but which I hope one day, maybe after my death, WILL be digitized, and will be incorporated into the computerized system that Martin Campbell devised for me in the 1990s and which simply digitized ongoing additions to the system I’d devised for myself in 1961. This was an entirely common-sensical system devised for myself, and not at all sophisticated. I was pleased to find, when reading Beatrice Webb’s My apprenticeship that in an appendix she’d recommended something rather similar to mine, but I can’t say that she influenced my system: it just confirmed me in operating the system I’d devised for myself.

I’ve always been rather surprised by three things: that (a) none of my seniors (not even Keith, or my research supervisor Peter Mathias) ever discussed research methods with me, nor did I discuss my research methods with anyone else. They didn’t seem interesting or original enough to discuss. For similar reasons, (b) I didn’t discuss them with my graduate students, though if they’d asked me I certainly would have done so. In retrospect, I was very ‘permissive’ with my graduate students, and simply left it to them to ask whether they felt any need for guidance, and I don’t recall any of them doing so. Absurdly, it would have seemed rather intrusive to venture into such territory: one assumed that they’d devise a system that suited them. Which leads me to say that (c) if there is anything particularly unusual about ‘my’ system, it’s that it has evolved continuously and (except for computerization) consistently, over more than half a century, so the driving force behind it is a curious combination of personal insecurity and ambition or perhaps megalomania, perhaps even a sort of secular substitute for the religious belief I never held strongly and fairly soon came to reject. As a young graduate student I felt insecure for two reasons: my mother’s family, far more influential upon me than my father’s, were anti-intellectual, and (understandably) couldn’t imagine where on earth historical research (let alone on temperance) could conceivably lead, and I was determined to ‘show them’. My father was really interested in what I was doing, but had re-married, and somehow we never talked much about it, and it didn’t occur to me that for him it would have been ‘real life’. My mother later told me that his father had addressed the public in Hull, or wherever in the north of England he lived, and had publicly advocated temperance standing above a public lavatory in Hull, but my father never talked to me about that, though I wish now that he had. Secondly, my parents were separating, and although both were very generous to me and ensured that I never lacked a home, in that situation I felt I really had to make a success of what I was doing. Self-help, central to my thesis subject, was therefore central also to my own life. Eventually, of course, the sheer interest of the research took over, and I needed no incentive – apart from vanity – to continue along the lines that I’d developed for myself. The coronavirus has now brought the newspaper dimension of all that to an end, but that had to end SOMEtime, and the other parts of the system will continue until… when? After this general introduction I embark in more detail upon the nine phases involved in developing my computerized database. Like so many postgraduates embarking on research in the 1960s I built up a card-index, initially for the purpose of writing the D.Phil. thesis (1966) from which came my ‘Drink and the Victorians. The Temperance Question in England 1815 1872’ (1971, 2nd. ed. 1994).

1. At first the card-index was concerned only with material relevant to the thesis, and consisted of bibliographical and factual cards – the latter derived from work in archives, references to books read and from any other type of source consulted while writing my thesis. As I realized the wide ramifications of my research subject, the database opened out into all sorts of adjacent subjects: nonconformity, feminism, recreation, pressure groups, the drink industry, industrialization, the Liberal Party, and so on. I typed up the cards on a typewriter, usually on only one side of each card, the size of a visiting card, beginning with a concise heading so that its contents could be quickly identified. At this formative stage I was much influenced by Beatrice Webb’s excellent essay on ‘The art of note-taking’, an appendix to her autobiographical ‘My apprenticeship’ (1926), which I came across by accident. My card-index was sub-divided from the start, and I increasingly realized the wisdom of her injunction to confine each card to a single subject, so that the card-index could be maximally flexible.

2. As my historical interests broadened, and as it became clear that my researches were likely to continue for a lifetime, the database gradually launched itself into research subjects on nineteenth-century British history that were new to me. Beginning as a ‘Victorian’ I gradually moved forward in time, in parallel with my pupils, to teach and research into the whole of British history since the early nineteenth century, so that towards the end of my life I have become a historian of contemporary Britain, reaching the early 21st century in my publications. I therefore ventured into such subjects as Chartism and pressure-group history, and through publishing two books on women’s history I moved into the Edwardian and inter-war periods, and became more political in my overall interests. My involvement in writing about the history of Oxford University took me forward to 1970, and in lecturing I was moving into the history of British political institutions, on which I was then teaching and lecturing; I published a book on them in 1996, The transformation of British Politics, a title suggested by the Press which I never liked, but which the Press suggested for lack of anything better. After retiring from the ‘ODNB’ in 2004, I moved into publishing on British history since 1951, writing in a more synoptic mode than earlier. Behind all this lay a digitized database that was continually growing in chronological reach and subject range. Subjects developed sub-subjects which then budded off into subjects in their own right, and these themselves developed sub-subjects, so that instead of only one typed-up card, I needed an increasing number of duplicates to be filed in the growing number of sections and sub-sections. I’ve already drifted into writing as though the card-index was in some sense organic, with a life of its own, which in a sense was the case. Like Harriet Beecher Stowe's Topsy, the card-index spontaneously "just growed".

3. At first I typed up the duplicate cards myself, but this soon became very laborious, and I brought the College’s xerox machine into service, so that selected cards could be xeroxed on to foolscap or A4 sheets, then guillotined into shape and filed. However, with a growing number of sections and sub-sections, and with almost everything bearing some relation to everything else, the number of duplicates required grew to more than one, and sometimes to as many as six or seven. So much so, that now when I come across a database record with only one code I look at it suspiciously, wondering whether there’s been an oversight. No scope here for fossilized academic ‘subjects’! With the one-card one-subject formula my intellectual world could be infinitely flexible, like the world at large. Filing the cards, however, became increasingly time-consuming and tedious, especially as with each card I had to decide how many copies of it would be needed, and this required me to decide where each card needed to be filed.

4. By the 1970s I concluded that there was nothing for it but to write a code on the bottom of each individual card before copying it, so that I could know how many copies were required and where they should be filed. This entailed creating a code-book which (like the card-index itself) grew and grew, with its pages often re-typed as required. This worked well enough, but with three very boring consequences: the cards had to be coded, then copied, then filed in the appropriate places. Preparing, coding and filing the cards became more and more time-consuming as the card-index grew. Less boring, however – even rather encouraging – was the fact that when the database was used to record new items, or to re-sort in aid of a publication, I had reason to trawl the database as a whole, and tidy it up with corrections, re-codings, supplementary codings or ideas for new divisions and sub-divisions – always, of course, looking out for typos and larger mistakes, which were numerous. In this way, the database was continuously not only growing but improving. Sorting through it was stimulating, too, because ideas often come from unexpected juxtapositions, and the whole system made such juxtapositions frequent.

5. by the early 1990s I could foresee that technology would soon come to the rescue – a rescue by then sorely needed. With the advent of hard disks in the mid-1980s all my typed-up material could in theory be stored up on a computer file instead of (as earlier) discarded once the card had been created. I predicted to myself that it ought soon to be possible to transfer this material from my files into a computerized database. A small step towards this was already in place by 1992, when Marion Ellis, who ran the computing unit in Corpus Christi College, helped me to devise a sub-divided template on to which my typed-up material could be collected for xeroxing, then guillotined and filed as before. By that time I’d made the painful transitions between operating systems: from TEXD to Wordstar, from Wordstar to Wordperfect, and then from Wordperfect to Microsoft Word.

6. by then, undergraduates were beginning to teach their ‘tutors’ about how to organize their lives better, and in 1995 the boredom was removed from my research procedures by a clever undergraduate who had taken business studies at school, and was now a second-year undergraduate reading engineering. He was Martin Campbell, who has since gone on to create and sell several computer companies, so I was lucky indeed. The Access database that he customized for me, devised to my specification, still admirably meets my needs quarter of a century later, and has never let me down, and rather to my surprise I gather that very few modifications to the Access database have been made in the past quarter-century, though I suspect that many improvements in the design of Access database facilities have been made during that interval. Mine was then the need, Martin had the intellect and the skills that I needed, and with his ready responsiveness to customer requirements he gave me exactly what I wanted. The computerized database has of course many facilities that the card-index never had. It’s infinitely flexible, its data is safer because it can be copied on to memory pens at home and on to storage facilities elsewhere, and its search facilities (word searches, spellchecks, code-searches, and combinations of the two) just didn’t exist before. Furthermore I can duplicate records whenever required, print out at will selected records in lists which are especially useful for bibliographies on selected subjects, and (as in any digitized text) move material around until it becomes clear and coherent. And instead of needing to type up or xerox copies of any entry, I need only once to assign the codes on any record that needs to appear on more than one section or sub-section.

7. by this time I had two data-collection systems running in parallel: the paper-based card-index and the newly-created computerized database, into which Martin was able to transfer all the material I’d entered up on cards since 1992. At that point, Martin asked one of those innocent questions from an outsider which turn everything upside down: why, he asked, did I need a paper-based database at all? When I thought about it I realized that, so long as I could trust the Access database, there was no need for paper at all. I took the plunge, and I have never regretted having done so. Whereas by 1995 I’d filled a battery of steel drawers with typed-up cards, occupying far too much space in our house and promising to fill many more in coming years, I had now reached what was technologically a stationary state, though since 1995 the database has continually grown and improved so that it now houses (in virtually no space at all) 315.737 records; furthermore these must represent what would have been (say) five times as many cards under the paper-based system: that is 1,578,685 cards. When editing the ‘ODNB’ in 2000-4, I brought my index into my office with me, leaving us much more space at home, and found both card-index and digitized database invaluable when writing ‘ODNB’ articles of my own and monitoring those by others. This was because collectively the database was by then so wide-ranging, but also because even its card-index section made the data so readily accessible, let alone the digitized section which was by then its only growth-point. Indeed, I made the database available to the research editors, but few of them used it, partly because so busy with the texts they’d already received, but also perhaps because when viewing it from a distance they found it more intimidating than it really was.

8. the next phase I have long yearned for, but it still eludes me. It would free our garage from its array of little-used (because difficult to use) cards kept in steel drawers containing my pre-1992 card-index. Purely from the space point of view, the card-index section of my database has become something of an albatross, though quite the reverse of that from the point of view of the valuable information that it contains. My final yearned-for phase is therefore the digitization of the 300,000 cards I created before 1992. Early on, we experimented with scanning in record cards, but this hoped-for remedy was too clumsy and time-consuming. There seems no escaping the fact that somebody will need to type into the database the card-index’s individual cards. Many of the cards were xeroxed, as explained in (2) above, and it might be possible to weed out the xeroxed items to reduce the size of the task. I lack the time to attempt this huge phase 8 myself, and I lack the funds to pay others to do it, and unfortunately (in the arts and humanities, at least) research-funding bodies tend to fund only the end-product, not the mechanism for getting there. Getting the job done would-300,000, or (to judge from my limited experience of an admirable firm in India some years ago) about a quarter as much if digitized by the army of educated processors of data who are to be found there.

9. one final benefit of a digitized database should be mentioned. It makes research – and, still more, writing – much more of a pleasure. By making it possible to find things easily, and to group material flexibly, it conveys a freedom of movement not possible before. It also removes the tedium arising from the need to re-type multiple copies of text. The pleasure of this can be enjoyed only by people of my generation; they alone can fully relish the change because only they can recall how laborious it was until the 1990s to prepare a publishable text. Now the digitized database requires its text to be typed in only once, though that one text can of course be infinitely refined and improved if desired. And because so much discovery and creativity results from insights prompted by encountering unexpectedly illuminating juxtapositions, the flexibility of the digitized database becomes central to the research process. What gets books written is getting into a situation where their author finds it exhilarating to work on them. With a database of this kind, as with digitized texts of any kind, things seem during the writing process almost spontaneously to 'fall into place' - whether it is a matter of experimenting with and arranging the text or of juxtaposing ideas from different directions. And because the process of discovery continues to the very end of the text’s preparation, the author sustains the energy and enthusiasm that makes it possible at last to reach the desired destination.