Accelerator Update for March 20, 2009 - March 23, 2009

The Integrated Luminosity 3/16/09 to 3/23/09 was 60.34 pb^-1.

Friday March 20

The day shift began with the Tevatron (TeV) in shot setup, with Operations monitoring the Recycler stash (389.9E10), with the Antiproton Source stacking (35.3E10), with the Linac, Booster, and Main Injector (MI) operating normally, with the MiniBooNE and NuMI experiments taking beam, and with the Collider Detector at Fermilab (CDF) and D-Zero (D0) experiments waiting for the next store.

Water experts reported at 8:09 AM that they had made up water for the TeV LCW system.

An RF expert reported at 8:17 AM that he brought a TeV RF station (TRF2) online.

Operations established store 6908 at 9:42 AM with an initial luminosity of 319.81E30.

Operations established beam to the Meson MTest experiment at 10:17 AM.

At 10:30 AM, a Linac RF station (LRF3) tripped off. It reset.

Recycler experts began a study period at 10:32 AM.

At 12:24 PM, a Booster RF station (BRF8) tripped off on a ground fault indication. It would not reset. Operators bypassed the station and brought BRF14 online in its place.

Operations prepared Booster for access at 1:11 PM to investigate the BRF8 ground fault.

Booster RF experts reported at 1:50 PM that they found a damper blower fan leaning against the RF bus bar.

Booster came out of access and Operations established beam to Pbar, MI, MTest, MiniBooNE, and NuMI at 2:26 PM.

Operations reported at 4:02 PM that the Recycler had stashed antiprotons four times during the day shift.

Operations disabled the MTest beam at 6:05 PM.

At 6:44 PM, BRF9 tripped off. It reset.

At 7:01 PM, BRF3 tripped off for the first of three times in an hour. It reset.

At 8:02 PM, LRF3 tripped off. It reset.

At 8:28 PM, BRF4 tripped off. It reset.

At 8:32 PM, BRF5 tripped off. It reset.

At 9:16 PM, a Booster RF expert investigated the many spark-induced trips.

At 10:22 PM, BRF9 tripped off. It reset.

Operations reported at midnight that the Recycler had stashed eight times during the evening shift.

Saturday March 21

The midnight shift began with Operations monitoring store 6908 (81.49E30) and the stash (340.1E10), with Pbar stacking (23.1E10), and with NuMI and MiniBooNE taking beam.

Operations terminated store 6908 at 1:09 AM.

Operations put the TeV into shot setup at 1:23 AM.

At 2:29 AM, BRF4 tripped off. It reset.

At 2:35 AM, BRF5 tripped off for the first of two times in an hour. It reset.

Operations established store 6909 at 2:41 AM with an initial luminosity of 309.85E30.

At 7:26 AM, the Pelletron tripped off. It reset.

Operations reported at 7:55 AM that the Recycler had stashed seven times during the midnight shift.

At 9:05 AM, BRF5 tripped off for the first of two times in 35 minutes. It reset.

At 9:16 AM, Operations reported that the I- Source had a jump in voltage. Operators put the device in local and adjusted it back to its normal output.

At 9:44 AM, the 400 MeV area beam permit tripped off due to bad beam positions.

At 10:26 AM, a Linac Klystron RF station (KRF6) tripped off. It reset.

At 12:18 PM, BRF5 tripped off for the first of two times in 30 minutes. It reset.

Operations terminated store 6909 at 3 PM.

At 3:09 PM, a TeV low beta quadrupole power supply (C:D0Q2) tripped off with a ground fault indication. Operations contacted experts.

Operations prepared D0 for an access at 3:21 PM.

Operators reported at 3:45 PM that they reset D0Q2, per an expert’s instruction, but the supply tripped off again. Operations called in power supply experts.

At 4:42 PM, a power supply expert reported that there was a short in the TeV tunnel for D0Q2. Operations began preparing the TeV for an access.

Operations reported at 4:57 PM that the problem with D0Q2 went away. Operations continued with the access preparation and called in TeV experts.

At 5:29 PM, CDF reported that they needed an access to replace a power supply that just failed. Operations prepared the area for access.

CDF came out of access at 7:12 PM.

TeV and power supply experts began their hunt for the D0Q2 ground fault at 7:33 PM.

Operations reported at 7:49 PM that the Recycler had stashed eight times during the last 12 hours.

Operations reported at 8:50 PM that the TeV access was over. Experts had found and fixed the D0Q2 ground fault. The pictures below and to the right show a cable that had its insulation rubbed away and where it shorted.

Experts repaired the cable and successfully conducted a hipot.

At 9:03 PM, BRF9 tripped off for the first of two times in about two hours. It reset.

Operations put the TeV through a dry squeeze at 10:50 PM.

At 11:22 PM, Operations reported that a TeV power supply D0Q5 had tripped off. It reset.

Operations put the TeV through another dry squeeze at 11:40 PM.

Sunday March 22

The midnight shift began with the TeV in a dry squeeze, with Operations monitoring the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.

Operations put the TeV through a wet squeeze at 12:22 AM.

At 1:14 AM, Operations reported that D0Q5 tripped off again. Operators investigated and reported a smell of burnt electronics.

Operations contacted a power supply expert about D0Q5 at 1:48 AM. He suggested that operators swap cards for the VFC, restore parameters, and zero the clocks and scalars. The power supply reset.

At 2:21 AM, a TeV quadrupole magnet (QFA4) tripped off its bulk power supply wall breaker. Operators investigated and then contacted the Duty Electrician about the 480-volt wall breaker.

At 2:38 AM, Operations reported that QFA4 had tripped off after a D0Q5 trip. Operations called in power supply experts.

At 3:07 AM, Operations reported a flurry of Booster RF trips.

Operations reported at 4:14 AM that the QFA4 power supply had internal damage. Power supply experts reported that the repair would be straight forward, but discovering why it failed would take a while.

At 5:26 AM, the Linac Klystron buncher tripped off. It would not reset. Operations called in Linac experts.

Linac experts had the Klystron buncher back online by 6:46 AM.

TeV power supply experts reported at 7:55 AM that they had found a 480-volt resistor board for QFA4 had arced and caused the AC breaker to trip. They replaced the board, various fuses, and SCRs. The supply was up and operating.

Operations reported at 8 AM that the Recycler had stashed once during the last 12 hours.

Operations put the TeV through a dry squeeze at 8:20 AM.

At 8:27 AM, the 400 MeV area tripped off. It reset. This was due to a Linac quadrupole power supply’s (QPS513) output current wandering around. Operators put the supply into its shunt mode.

At 8:57 AM, the 400 MeV area tripped off. It reset.

Operations put the TeV through a wet squeeze at 9:11 AM.

Operations put the TeV into shot setup at 9:39 AM.

At 10:48 AM, the 400 MeV area tripped off. It reset.

Operations established store 6916 at 10:56 AM with an initial luminosity of 303.34E30.

At 6:10 PM, Operations reported that about 7 mA of antiprotons were lost during a transfer when a Recycler injection kicker (KPS2A) failed to fire at the correct time. The rest of the stash went okay.

Operations reported at 7:55 PM that the Recycler had stashed nine times during the last 12 hours.

At 9:51 PM, the 400 MeV area tripped off. It reset.

Operations terminated store 6916 around 10:36 PM.

Operations put the TeV into shot setup at 10:53 PM.

Monday March 23

The midnight shift began with the TeV in shot setup, with Operations monitoring the stash, with Pbar stacking, and with NuMI and MiniBooNE taking beam.

Operations established store 6917 at 12:10 AM with an initial luminosity of 325.28E30.

At 1:06 AM, LRF3 tripped off. It reset.

At 1:26 AM, the 400 MeV area tripped off for the first of four times in about two hours. It reset.

At 4:52 AM, the 400 MeV area tripped off for the first of 19 times in about two hours. Operators halted the Linac beam and called in experts.

Linac experts brought Linac back online at 7:38 AM, but at a lower intensity.

Operations reported at 8 AM that the Recycler had stashed 10 times during the last 12 hours.

The Plans for Monday

The plans for today are to stack, stash, and store. The Run Coordinator will allow Recycler experts to test a new way of extracting only a certain predetermined portion of their stash. Linac experts will tune.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.