Accelerator Update for March 4, 2009 - March 6, 2009

Wednesday March 4

The day shift began with Tevatron (TeV) experts investigating the Tuesday night quench, with Operations monitoring the Recycler stash (222.6E10), with the Antiproton Source stacking (15.5E10), with the Linac, Booster, and Main Injector (MI) operating normally, with the MiniBooNE and NuMI experiments taking beam, and with the Collider Detector at Fermilab (CDF) and D-Zero (D0) experiments waiting for the next store.

Operations put the TeV through a wet squeeze at 8:14 AM.

At 8:27 AM, a Linac RF station (LRF3) tripped off. It reset.

Operations put the TeV into shot setup at 8:34 AM.

At 9:09 AM, Operations received a report of a LCW leak from a TeV sector A2 power supply. An expert investigated.

At 9:31 AM, Operations received a high alarm from an Accumulator power supply (A:IB). An expert investigated and determined that it was a controls problem.

Operations turned off the NuMI beam at 10:10 AM so an expert could access MI-65.

A Power supply expert reported at 10:32 AM that the TeV sector A2 leak was not from a power supply, but from a leaking hose in the dump rack and it was dripping on a SCR. He replaced the hose and dried out the SCR.

At 10:47 AM, TRF3 tripped off. An expert investigated.

Operations reported at 10:53 AM that after the TeV sector A2 dump leak was repaired, operators successfully completed a dump test.

At 10:54 AM, BRF3, 8, and 15 tripped off due to sparking. They all reset.

At 10:59 AM, Operations reported that a Linac Klystron Debuncher was holding off beam. Experts investigated.

Controls experts reported at 11:10 AM that they had replaced the MADC for A:IB, A:LQ, and D:IB.

Operations put the TeV through a dry squeeze at 11:18 AM.

Linac experts reported at 11:26 AM that they found and replaced a bad relay on a LCW valve. Linac resumed sending beam.

Operations began a new TeV shot setup at 11:41 AM.

A NuMI expert reported at 12:03 PM that he had added water to horn #2. He reported that they had found some damage to the Argon flow meter on horn #1, but it was still functional. They will order a new meter.

Operations established beam to NuMI at 1:07 PM.

At 1:20 PM, Operations reported that the MI-20 LCW pump #1 had tripped off. Operations informed experts who restarted the pump.

Operations established store 6862 at 1:31 PM with an initial luminosity of 257.3E30.

At 2:09 PM, the P1 transfer line beam permit tripped off when a power supply (I:V714) drifted low. Everything reset.

At 2:14 PM, Operations reported that BRF1 had tripped off twice. Operations contacted experts who bypassed the station.

A MI expert began a 12-minute flying wire study in the MI-8 transfer line at 3:48 PM.

Operations reported at 4:07 PM that the Recycler had stashed antiprotons four times during the day shift.

Booster experts reported at 4:16 PM that they had brought BRF1 back online.

At 4:41 PM, Operations reported that a Linac quadrupole power supply (L:QPS114), which had tripped off a short time ago, tripped off again and caused QPS113 to also trip. A Linac expert investigated, brought the supplies back on, and beam returned.

At 5:52 PM, L:QPS113 tripped off again. Operations contacted an expert who suggested operators swap out control cards.

At 6:17 PM, a Booster vacuum pump (IPRF23) tripped off and caused BRF13 and 14 to also trip. Everything reset.

At 6:43 PM, a Booster vacuum pump (IPRF25) tripped off and caused BRF15 and 16 to also trip. Everything reset.

Operations reported at 7:10 PM that operators had swapped the QPS113 cards with QPS110. Linac resumed sending beam.

At 8:29 PM, BRF1 tripped off for the first of two times in five minutes. It would not reset after the second trip. Operators bypassed the station.

At 10:43 PM, operators tried to bring BRF1 back online, but it tripped off and caused BRF3 to also trip. BRF3 would not reset.

At 11:43 PM, BRF1 tripped off again. BRF3 was still down. Experts came in to investigate.

Operations reported at midnight that the Recycler had stashed six times during the evening shift.

Thursday March 5

The midnight shift began with Operations monitoring store 6862 (80.29E30), the stash (223.5E10), and the Pbar stack (23.9E10), and with no beam to NuMI and MiniBooNE due to Booster RF problems.

At 12:38 AM, Operations reported that Booster beam had resumed briefly, but then BRF4 tripped off.

Booster resumed sending beam at 12:48 AM. The experts reported that they had replaced the bias power supplies and left BRF3 bypassed.

At 2:01 AM, L:QPS113 tripped off. It reset.

Operations terminated store 6862 at 6:52 AM.

Operations put the TeV into shot setup at 7:04 AM.

At 7:08 AM, Operations reported that a Booster poser supply (B:VL21) had tripped off. (A poser supply is not a real power supply but another piece of equipment posing as a power supply because of the prestige.) Operators replaced the power supply.

Operations reported at 7:58 AM that the Recycler had stashed six times during the midnight shift.

At 8:11 AM, Operations reported two beam permit trips for the 400MeV area due to bad beam positions. It reset.

At 8:12 AM, Operations reported two more trips for the 400 MeV area. It reset both times.

At 8:25 AM, the 400 MeV area had tripped off four more times. Experts investigated and reported that the 400 MeV chopper “on-tube” had failed. Operations turned off the Linac beam for this repair.

Operations established store 6863 at 8:48 AM with an initial luminosity of 324.57E30.

At 8:50 AM, BRF3 tripped off. An RF expert investigated.

Fire Department technicians reported at 9:13 AM that they were beginning to test the fire systems for the MiniBooNE and MI service buildings.

At 9:23 AM, L:QPS113 tripped off. An expert investigated.

At 12:33 PM, BRF3 tripped off for the first of three times in about an hour. After the last trip, operators bypassed the station and brought BRF14 on in its place.

At 2:14 PM, BRF6 tripped off. It reset.

At 2:26 PM, BRF4 and 9 tripped off. They reset.

At 2:42 PM, a MI expert was looking at coalescing and noticed a variation in the amount of beam making it into MI. She reported it as a transition fallout. A Booster expert investigated and made adjustments.

CUB experts reported at 3:19 PM that the repair of the Booster LCW pump “B” was complete. It will be brought on line when an opportunity appears.

At 3:29 PM, LRF3 tripped off. It reset.

Operations reported at 3:57 PM that the Recycler had stashed five times during the day shift.

A Booster RF expert brought BRF3 back on line at 5:35 PM.

At 6:43 PM, BRF3 tripped off twice. Operators bypassed the station and contacted an expert.

A Booster RF expert brought BRF3 back on line at 11:32 PM.

Operations reported at 11:57 PM that the Recycler had stashed eight times during the evening shift.

Friday March 5

The midnight shift began with Operations monitoring store 6863 (76.51E30) and the stash (334.2E10), with Pbar stacking (21.6E10), and with NuMI and MiniBooNE taking beam.

Operations terminated store 6863 at 2:04 AM.

Operations put the TeV into shot setup at 2:15 AM.

At 3:24 AM, the TeV suffered a quench at sector D0 during the squeeze. Operators investigated and contacted a TeV expert.

Operations started ramping the TeV at 6:53 AM.

At 6:55 AM, the TeV suffered a quench at sector F3.

A TeV experts suggested at 7:42 AM that there was a QPM failure. Operators discovered that two HFUs showed an address fault. Operations contacted a controls expert.

The Plans for Friday and the Weekend

The plans for today are to resolve this TeV quench problem and get back to normal stacking, stashing, and storing. The Run Coordinator will allow a MI power supply swap and a Pbar study.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.