Monday November 14
The day shift began with Operations monitoring store 4499 and the Recycler stash, with the Antiproton Source (Pbar) stacking, with the Collider Detector at Fermilab (CDF) and D-Zero (D0) using the colliding beam, with MiniBooNE down due a bad power supply (H866), and with NuMI and Meson MCenter experiments taking beam.
Operations prepared MiniBooNE for an access at 8:36 AM.
At 9:35 AM, two Main Injector (MI) RF stations (MIRF8 & 18) tripped off. They both reset okay.
A power supply expert reported at 10:09 AM that the MiniBooNE H866 power supply had a bad transformer.
Operations began preparing the Recycler to stash antiprotons at 10:19 AM.
Operations reported at 12:02 PM that work on H866 had been completed and operators were sending beam to MiniBooNE.
The Recycler stash ended and Pbar resumed stacking at 12:28 PM, but just to verify everything was okay. Operators then prepared the Pbar rings for a safety system test.
Operations shut down all beam at 1:27 PM for transport safety system testing.
The Pbar safety system testing was completed at 1:30 PM. Operations began preparing the rings for an access.
At 3:56 PM, Operations reported that they were having trouble getting Booster on. An operator contacted an expert.
At 4:06 PM, an expert began investigating why Operations weren't receiving any alarms.
At 4:10 PM, Operations discovered that two Linac RF stations (KRF6 & 7) were off. Operators had KRF7 up in two minutes.
At 4:10 PM, operators reported problems with two special MiniBooNE LCW systems. An operator called an expert.
At 4:21 PM, Operations reported that KRF6 was still down. This held off all downstream beam. Linac experts investigated and found a leaking LCW fitting in the station. They also found a leak from the same fitting in KRF1. They began repairs.
A MiniBooNE expert reported at 4:29 PM that there was an instrumentation problem with the special LCW systems. Water experts continued to investigate.
At 4:50 PM, Operations reported that the NuMI beam permit dropped on a Target and Baffle water flow. The trip would not reset. Experts investigated.
Operations reported at 4:50 PM that the Main Control Room's alarm display problem was resolved by a reboot.
KRF6 came back on line at 5:04 PM.
Operations established beam to MiniBooNE at 5:14 PM.
A water expert increased the flow at NuMI, which cleared the alarm. NuMI resumed taking beam at 5:58 PM.
The Pbar safety system tests ended and Pbar resumed stacking at 7:41 PM.
At 9:14 PM, KRF5 tripped off. It reset okay.
At 11:27 PM, BRF18 tripped off and wouldn't reset. Operators bypassed the station.
Tuesday November 15
The midnight shift began with Operations monitoring store 4499 and the stash, with Pbar stacking, and with MiniBooNE, NuMI, MTest, and MCenter taking beam.
At 12:09 AM the Recycler suffered a vacuum burst.
At 12:43 AM, MIRF8 tripped off. Operators reset the station.
Operations terminated store 4499 at 1:02 AM.
Operations prepared the CDF and D0 collision halls for an access at 1:20 AM.
Both CDF and D0 were out of access by 3:18 AM.
Operations put the TeV through a dry squeeze at 3:42 AM.
Operations put the TeV into shot setup around 4:10 AM.
At 4:16 AM, Operations reported that MIRF8 was repeatedly tripping off. Operations consulted with experts.
Operations reported at 5:28 AM that NuMI was off while operators worked with MIRF8.
Operations reported at 5:53 AM that Operators, with some expert suggestions, managed to bring MIRF8 back on line.
Operations established store 4501 at 6:47 AM with an initial luminosity of 124.48E30.
At 6:59 AM, Operations prepared MI for an access.
At 7:57 AM, Operations reported that the TeV beam positions were causing some problems and that the tune tracker for protons wasn't working. Operators made some adjustments.
Operations prepared the Meson beam lines for an access around 8 AM.
Operations began preparing the Booster for an access at 8:50 AM.
At 9:01 AM, the interlock techs working in MI reported seeing a good deal of water on the floor in MI-40. Operators called the Duty Mechanic who later installed a portable sump pump.
Operations prepared the Pbar rings for access at 9:21 AM.
Operations prepared the AP2 transfer line at 9:34 AM for an access to fix a vacuum problem. This access required Operations to dump the Pbar stack.
An expert reported that the AP2 transfer line vacuum leak was isolated at 1:19 PM. They're installing a vacuum pump.
Operations prepared MiniBooNE for an access at 1:35 PM.
A vacuum expert reported that the P1 transfer line vacuum valve work was completed at 2 PM.
Linac was prepared for safety system testing at 3:23 PM.
Operations established beam to the Linac dump at 5:10 PM.
Operations established beam to the Booster dump at 5:14 PM.
Operators pulled the portable sump pump out of MI-40 at 10:52 PM.
Operations established beam to MiniBooNE at 11:14 PM.
Operations established beam to NuMI at 11:55 PM.
Wednesday November 16
The midnight shift began with Operations monitoring store 4501, and with MiniBooNE and NuMI taking beam.
At 12:26 AM, Operations reported a MI LCW pump was off and operators halted the ramp to investigate.
Operations established beam to the Recycler at 12:54 AM.
Operations established beam to MTest and MCenter at 1:14 AM.
At 1:22 AM, Operations reported that an MI vertical sextupole power supply (SEXVPS) wouldn't stay on.
At 1:34 AM, the Pbar quadrupole shunts alarmed. Operators restored the shunts to their proper values.
A Recycler/ Pelletron expert began an Electron Cooling study at 1:59 AM.
At 2:02 AM, Operations reported that SEXVPS tripped and wouldn't reset. An operator contacted a power supply expert who said that this device was very sensitive to LCW pressure. Operations contacted experts.
At 3:31 AM, Operations reported that a TeV RF station (TRF4) tripped off, which caused luminosity to drop.
At 3:32 AM, Operations reported that NuMI was tripping off repeatedly.
Operations terminated store 4501 at 7 AM.
The LCW experts reported at 7:24 AM that they had resolved the MI-60 LCW pressure problems. SEXVPS had resumed normal operations.
The Plans for Wednesday
The plans for today are to complete our safety system checks. There will be a TeV tunnel access later today and a D0 and CDF access. The Run Coordinator hopes that Pbar will resume stacking this afternoon so that we can establish a store this evening.
Accelerator Update Archive
For Tevatron luminosity charts and the current status of Fermilab's
accelerators and detectors (live!), please go to Fermilab Now.
Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: firstname.lastname@example.org.