This page is about my own experiences of the TOPS(Total Operations Processing System ) project to computerise the wagon fleet management, with the aim of providing some background and technical information that complements the other information about TOPS currently available on the web.
In March 1975 I was working in the London Midland region signalling drawing office in Carlow Street, just round the corner from Mornington Crescent. British Rail used to circulate a weekly list of vacancies, and one week I saw a requirement for a junior system programmer for the new TOPS project for the wagon fleet management listed. As it said "no previous programming experience required" I applied without a clue of what a junior system programmer actually was or did.
After few weeks I was called for an interview, and had to do a pre-interview aptitude test in a room with about 30 other people. The test was all about numbers in numbered boxes. After about half the allotted time I had finished, and checked it over twice and was getting restless, when I noticed there were two other people in the same state, whilst the person next me didn't seem to be even half way through. So the three of us got up and handed in our papers. Interestingly enough we were the only three who were called back for a full interview and were all subsequently offered jobs.
I was one of the last people to join the project. The original staffing for the systems programming side of the house had drawn on the best available talent at other pre-existing BR computer centres at Swindon, Peterborough etc. but as none of these centres used IBM mainframes a number of non-BR people who were well versed in IBM software were also recruited. For the application programming teams people were likewise drawn from the existing computer centres, and also a number of railway operations people were trained up as programmers, so that the teams had a good cross section of experience. In addition a number of people from the general BR annual graduate intake were recruited.
The TOPS project HQ was based in Blandford House, in Melbury Terrace, London. It was a nondescript building next to Marylebone station with no external indication of purpose apart from a brass plate with just the address on it, and blacked out windows on the ground floor. The entrance to building was through solid wooden doors with an airlock like arrangement, and I was given a security card with associated pin number.
The communications equipment was on the ground floor, and the computer machine room occupied the whole of the first floor, which because of the shape of the building was a long rectangular space that looked very impressive when standing at one end and looking down the total length.
The second floor housed the management office and support services. All the programming staff were on the top floor which was one big open plan space with thick deep pile carpet of the same design as used in first class carriages of the time. In reality it was a converted attic with glass down one side which was very light and airy, but large area of glass was not good news in the long hot summer of 76. The building came to be known as the Blandford House Computer Centre (BHCC)
When I joined, the project had been under way for quite some time, with all the hardware up and running, along with the TOPS software ready to be rolled out, and the computer centre was fully staffed.
The hardware consisted of
2 X 168 OS/370 mainframe computers with
4 Mb of RAM storage each, and several banks of 3330 disk drives which used removable disk packs.
One mainframe was dedicated to running TOPS using the MFT operating system, and this was configured with three partitions:
The software behind MCJ/MEJ/MPJ was written in IBM Assembler (IBM Basic Assembly Language or BAL) for reasons of speed and size. Each job consisted of a number of tasks, so one of the tasks within MPJ was FRM (File request manager) and that handled all the file requests as TOPS used its own file structure with direct access, index and sequential files rather than a proprietary DBMS. The wagon file was a direct access files with a system of buckets, baths and reservoir in which each wagon had a record of around 320 bytes with a 10 byte key.
The application programming was done in TOPSTRAN which was a set of assembler macros that that hide the complexity of the underlying assembler code from the application programming team. There was a batch test program that simulated the online system, so that first cut testing did not an online test environment. The batch program was accompanied by a comprehensive set of test data, and it was setup so that the data was, by default, restored back after a batch test had finished, making it very easy and quick to test changes, against a predictable set of test data.
Meanwhile the offline machine used MVT/ASP and on a day to day basis it ran development and testing work, but it's prime role was to be available as a backup machine when the online machine had a hardware failure or needed to be taken down for maintenance. It should be remembered that TOPS was a "real-time, online" system to use the jargon of the day, but in modern parlance it would be called a 24 x7 (even including Xmas day) operation, and TOPS was probably one of only half a dozen such systems in the UK in the 70's. Unlike most mainframes at the time the online machine did not stop processing for overnight backups, and had a system of backups, log files, journals etc being spun off after midnight.
[ Various references on the web say TOPS was implemented in 1973 but I am pretty sure that when I started on the project in March 1975 it was not yet live and it was later in 1975 when the project started to go live. I think the 1973 date refers to the new locomotive numbering scheme which had been introduced to ensure unique numbering for every loco and wagon, and this page seem to indicate that the loco renumbering was only completed in 1975 as it states "the TOPS renumbering took a little time to gather speed but was completed by the latter part of 1975" . However TOPS was not the reason for renumbering as that was the result of a BRB CM&EE (British Railway Board Chief Mechanical and Electrical Engineer) initiative of the early 1970s known as the Unique Numbering Scheme, which had a great deal to do with their introduction of a mainframe system the acronym of which was (and is) RAVERS. ]
Each depot was originally mean to have a IBM card reader/punch but early trials found that these could not cope with the dusty conditions of the freight yards (think next door to coal mine for some of them) so instead each one had a mini-computer emulating the IBM card reader/punch. These still had cards but these were 120 column cards that were much smaller and were less sensitive to dusty conditions. The emulation was considered sufficiently novel by the Open University that it was featured in one of their computing modules. After about a year or so the minis were reprogrammed to use a system called "cardless TOPS" which effectively replaced the physical cards with virtual cards. This was a novel concept at the time and became the subject of an Open university computing module.
My first job was to produce a performance statistics reporting system written in assembler and PL/1. The assembler routine was used to access the data stored on a dedicated 3330 disk pack with one cylinder per days data ( a 3330 disk pack held 404 cylinders, conveniently just a bit bigger than the 366 required for a leap year).. No statistics package such as SAS existed at that time, so reports, histograms etc had to be hand crafted in PL/1.
By the mid-seventies the BR wagon fleet probably numbered some 400,000 wagons, and prior to TOPS was still being managed manually, so it is probable that no-one knew the exact size of the total pre-TOPS wagon fleet because of the large number of wagons that pre-dated nationalisation.
With a big project like TOPS a phased implementation made a lot of sense for the following reasons
The way in which TOPS went live was quite unusual as it was done geographically, starting in Penzance area and gradually rolling out slowly across the country with the tip of Scotland being the last to be live online. There was an invisible virtual line delimiting the TOPS enabled part of the UK from the non-TOPS rest of the country, and as wagons crossed over this line, those that were not already on TOPS were added to the system every time the operations staff came across them in freight trains. Additionally moving up the country in front of this invisible line were the training staff who trained people in the next area to be cutover so that the TOPS training could be spread out over time with the minimum number of trainers.
As the rollout of TOPS progressed, by the time it was approaching Birmingham, the technical planning manager, who was responsible for all the hardware and software, had done some extrapolations of system load, and it looked like a single 168 mainframe would not be able to cope with the workload once the whole country was cutover.
To ensure that adequate processing capacity was available an additional IBM 158 mainframe was procured and installed. The plan was that TOPS software would be updated so that the 158 would run the MCJ program and a 168 would run MEJ/MPJ, or in the event of the 158 failing the other 168 would cover for it.
No sooner was the 158 up and running and being tested, than the processor load started to stabilise. It became apparent that as each new area came onto the system, its transaction load was initially higher than anticipated because users would try things out by running additional enquiries etc, and also make more mistakes, and hence resubmit transactions ,so these two factors artificially increased the number of transactions per day. However over time the processor load then fell back to around about the original estimated workload as once users were comfortable with the system they made fewer mistakes and spent less time just trying things out.
Also when TOPS was being used in the USA it covered a number of times zones, but in the UK it was all in one time zone, so this compressed the morning and evening peak busy periods in to much shorter time windows. In the 70’s freight was mainly run overnight, often using the same electric or diesel locos that hauled mainline passenger trains during the day time with freight trains leaving between 23.00 and midnight and arriving around 06.00, so these two time slots were the peak busy period for TOPS activity.
Finally the system programming team for TOPS had identified, one by one, a number of improvements that could be made to the software to eliminate bottlenecks etc., and as each of those improvements was implemented the software became that little bit more efficient.
By the time the whole country had been cut over and the system had settled down it was handling some 200K transaction a day.
Things did not stand still after implementation, and the system itself had additional tuning to improve performance and various add-ons developed.
The data that was spun off overnight was put to good use, and one application was an offline historical database using IBM's IMS dbms and this was appropriately called TARDIS (TOPS ancillary retrospective database information system). Initially requests for information were done with paper forms, but that system was replaced by a new TOPS transaction which submitted a request for overnight processing overnight with the report being printed and sent out in the internal post, but eventually the output was routed back via TOPS to the person who had originally submitted it.
In the late seventies a IMS loco database, also using IMS/DB, was developed. The system requirements for this had to meet the needs of both the electric and diesel engineering departments, which could at times be quite different e.g. electric locos had a time based maintenance intervals and diesels were mileage based.
TOPS was a major culture change for railway operations, and there was initial quite a lot of scepticism, and when the system went down due the hardware failure the field staff referred to the manual paper system used in that situation as BOTTOMS (back on to the old manual system). TOPS was meant to ensure better utilisation of the wagon fleet so that departments such as Civil Engineering were meant to "pre-order" wagons for use on weekend engineering work, and it took a while for them to trust the system to schedule the ordered wagons to turn up on time rather than hoarding wagons during the week ready for the weekend.
The figure I heard quoted was that TOPS shrunk the wagon fleet down from 400,000 down to 200,000 wagons within two years of implementation with knock savings in maintenance costs and freeing up of space occupied by sidings for other use.
Round about 1977/78 the second 168 operating system and scheduler/spooler was upgraded from MVT/ASP to MVS/JES2 to take advantage of the much improved reliability and throughput that MVS offered. Meanwhile the research department at Derby had an IBM mainframe (a 145 ?) running VS/1 and this had a very cpu bound workload, which nicely complemented the I/O (input /output) bound workload of the Marylebone offline 168. So the Derby mainframe was decommissioned and the Derby user were given remote access to the MVS machine via two links for time sharing (28k bandwidth) and a RJE(Remote job entry) terminal and printer. In London an internal bureau service was offered on this mainframe to departments such as Operational Research, Strategic planning Civil engineering etc.
Also during this period the offline machine started running non-TOPS related tasks such as Carousel, a merry go round train fleet scheduling program which was a joint venture between BR, CEGB and NCB with input from the Met Office. This was used to help predict the coal and electricity demand for the coming week based on the likely weather, and achieve the required coal delivery to power stations with the minimum of train movements. As the TOPS programming activity dropped back the BHCC programming teams, and people were used on other activities such as helping to create a reservation system based on an ACP/TFP system acquired from the Corsican ferries that was eventually used on the East Coast mainline.
Eventually TOPS was moved from BHCC to the Nottingham data centre, Blandford House was demolished and the site is now occupied by the BNP Paribas building.
After TOPS had been running for a couple of years we had a series of system crashes, one every couple of weeks or so. I sat opposite the systems programmer responsible for diagnosing memory dumps from system crashes and he was very perplexed by these, which was unusual for him as he usually got to the bottom of things quite quickly. It almost got to be a regular Friday afternoon thing when he would haul one of these memory dumps (produced on paper in those days) onto his desk and would pore over the hexadecimal contents through a nicotine haze and mutter “this should not be happening” Then one day the TOPS mainframe went down with a hardware failure which we were told was due to a tiny, slow water leak over the CPU card. My colleague immediately threw out his three foot high paper archive of memory(core) dumps because his conclusion was that the water leak was causing oddball processing side effects in the CPU circuits and he was spot on as we never got any more of these errors.
Around the same time the application programming team had a software error reported where a train had en route between depots had one wagon mysteriously disappear but a totally different one had been added to the train. The operations staff out in the field were convinced it was a software error as in the 70's computers were still viewed with deep suspicion. Eventually the penny dropped with someone in the application programming team who asked “Can you check the wagon has the same number on both side” which, of course, it hadn’t . It must have been trundling round the railway networks in that state for weeks if not months.
Les Smith
December 2009
(updated last : April 2010)
British Transport Films made a number of documentaries on TOPS as below, but none have yet made their way onto the DVDS released by BTF