Operation Summary for July 18, 2011 - July 20, 2011

Monday July 18

The day shift began with NuMI off due to target replacement, with Tevatron (TEV) personnel investigating 150 GEV extraction problems, with Operations monitoring the Recycler stash (390.6E10), with the Antiproton Source (Pbar) stacking (120.4E10), with the Main Injector (MI), Linac, and Booster operating normally, with the MiniBooNE experiment taking beam, and with the Collider Detector at Fermilab (CDF) and D-Zero (D0) waiting for the next store.

During the day shift, the MI-10 kicker room air conditioning (AC) tripped off at 8:19 AM. FESS personnel reset it. At 8:23 AM, Cryo System technicians reported that the TEV sector D1 wet engine flywheel bearings were failing; they began repairs. An RF person brought a MI RF station (MIRF5) back online. Operations disabled beam to MiniBooNE, at 8:31 AM, due to high temperatures in the MI-12 service building. TEV personnel reported at 9:30 AM that the TEV reverse proton problem was due to the TEV being on the wrong helix. At 10:18 AM, a TEV quadrupole magnet (AQ9) suffered a quench/ antiquench; an EE Support person reported it was due to a current transductor glitch; it took him about an hour to replace the transductor. D0 reported at 10:48 AM that they were conducting cosmic ray studies. Operators prepared CDF and D0 for access at 11:22 AM. Cryo System techs reported at 12:12 PM that they had completed their work on the D1 wet engine. At 1:20 PM, another TEV quadrupole magnet (BQ0) showed a quench indication; EE Support personnel investigated and reported that it had the same problem as AQ9 and replaced its transductor. FESS reported that they had repaired the TEV F4 AC unit. CDF and D0 came out of access at 2:13 PM. Operators completed a low beta magnet check of the TEV at 2:54 PM. FESS personnel reported at 3:06 PM that they had replaced the fan motors for the big AC unit at MI-12; they also installed another portable AC unit. Operators put the TEV through a dry squeeze at 3:43 PM. A MiniBooNE person reported at 3:52 PM that the MI-12 temperature was good enough to resume sending beam to MiniBooNE. Operations established beam to MiniBooNE at 4 PM.

During the evening shift, Operations put the TEV into shot setup at 4:53 PM and established store 8903 at 6:08 PM with an average initial luminosity (AIL) of 385.5E30. At 6:59 PM, operators reported that the MI-8 transfer line was experiencing a lot of beam aborts; operators put Pbar into standby mode and investigated. They found and replaced a bad control card.

Tuesday July 19

The midnight shift began with Operations monitoring store 8903 (143.01E30) and the stash (295.6E10), with Pbar stacking (27.7E10), and with MiniBooNE taking beam.

During the midnight shift, at 2:45 AM, all the Tevatron Electron Lens (TEL2) correctors tripped off; everything reset. At 3:03 AM, TEL2 tripped off again; it reset. Operators reported conducting a Booster injection tune at 3:33 AM. Around 4:39 AM, Operations terminated store 8903, put the TEV into shot setup, and established store 8904 at 5:49 AM with an AIL of 408.11E30. Operators reported short downtimes due to LRF1 & 2, due to two Linac Klystron RF stations (KRF3 & 5), and due to a Booster RF station vacuum pump (IPRF17).

During the day shift, a Booster person conducted a 25-minute damper study at 9:54 AM. Safety System personnel conducted a 20-minute pressure test of the MTA cavity at 10:59 AM. Operations disabled beam to MiniBooNE at noon for AC work. Operations began sending beam to MTA at 12:41 PM for studies. Operators reported that a three Recycler corrector power supplies stopped working due to a bad control card; operators asked Controls personnel to change out the card, but were told they would lose the stash. Operators will transfer all the antiprotons out of the Recycler and then change the card. Operations disabled beam to MTA at 1:40 PM. MI personnel began a g-2 study. At 3:10 PM, store 8904 aborted due to a problem with a TEV quadrupole power supply (C:BQ0); EE Support personnel investigated. FESS personnel reported that they had dealt with numerous AC problems throughout the shift.

During the evening shift, Operations established beam to MiniBooNE at 4:19 PM. Operators turned off the TEV ramp at 4:49 PM for C:BQ0 repairs. At 5:11 PM, a MI power supply (I:VTF11) tripped off and would not reset; operators contacted EE Support for help. Operations established beam to the new Meson FTBFT-1008 ‘SuperB Muon Detector Prototype’ experiment. The Cryo Coordinator reported at 5:32 PM that the cryo system at TEV sector F1 was struggling due to the heat load from the cold compressors; they needed to shut off the cold compressor for an hour to let it cool down. At 5:33 PM, operators reported high temperatures at the MI-10 service building and that the BPMs were not responding; FESS personnel investigated and said that the AC unit #2 was off. They power washed the AC coils and brought it online. Operators prepared D0 for access at 5:47 PM. Operators reported that EE Support had completed repairs on I:VTF11 and Pbar resumed stacking at 6:38 PM, but that the high temperature at F1 was the real problem for the failure. FESS personnel investigated and cleaned out the F1 AC air intake. D0 came out of access at 7:08 PM. Operators reported that they had put the TEV through a dry squeeze and a wet squeeze by 9:37 PM. FESS personnel reported that they installed a portable AC unit in MI-10 at 9:40 PM. At 9:41 PM, a Pbar Debuncher power supply (D:H914) tripped off on a phase unbalance fault; operators called in a Pbar power supply person. Operations disabled beam to T-1008 at 9:50 PM. Operations put the TEV into shot setup at 9:58 PM. Operators reported that during the reverse proton setup for the shot setup, the helix didn’t seem to be in the correct state; a TEV person had operators reset DCE26 and this fixed the problem. A Pbar person reported at 10:54 PM that he had replaced the phase unbalance detector for D:H914; Pbar resumed stacking. Operations established store 8907 at 11:24 PM with an AIL of 330.3E30.

Wednesday July 20

The midnight shift began with Operations monitoring store 8907 (255.23E30) and the stash (70.8E10), with Pbar stacking (14.5E10), and with MiniBooNE taking beam.

During the midnight shift, operators reported at 12:33 AM that there were several TEV LCW high temperature alarms and that the F1 temperature had reached 109˚ F before it started decreasing. At 12:53 AM, the Booster West Anode Power Supply tripped off and BRF7 also tripped off, but would not reset. Operators bypassed the station and brought BRF19 online in its place. Operators reported at 1:21 AM that the kicker room temperature at MI-10 was coming down. At 1:33 AM, operators reported that the beam permit for the Linac 400 MeV area tripped off; operators investigated and reported a problem with the 400 MeV chopper. Operators called in a Booster person to investigate the chopper problem. Operators put Pbar into standby. Booster resumed sending beam at 3:59 AM. Operations established beam to T-1008 at 4:38 AM. At 4:59 AM, the MiniBooNE horn power supply tripped off and would not reset; operators called in power supply technicians. They reported at 7:05 AM that they traced the horn problem to a control chassis at MI-12, which they replaced.

The Plans for Wednesday

The plans for today are to stack, stash, and store. However, the Run Coordinator will allow a Pbar rings access this morning to search for a LCW leak. She will also allow a TEV study this evening. NuMI personnel report that they will conduct target scans this afternoon and, hopefully, resume taking normal beam tomorrow, 7/21/11.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.