Operation Summary for July 20, 2011 - July 22, 2011

Wednesday July 20

The day shift began with Operations monitoring the Tevatron (TEV) store 8907 (98.73E30) and the Recycler stash (196.8E10), with the Antiproton Source (Pbar) off to search for a LCW leak, with Booster off for water turbine work, with the Main Injector (MI) and Linac operating normally, with the MiniBooNE experiment off due to Booster work, and with the Collider Detector at Fermilab (CDF) and D-Zero (D0) using the colliding beams.

During the day shift, a Pbar person reported at 8:43 AM that he found the LCW leak on an Accumulator sextupole magnet (A5S12); Water Works personnel began repairs. Operations reported at 9:30 AM that they had received permission to send beam to NuMI for a target scan study. Linac personnel reported at 9:37 AM that they had begun repairs on their Debuncher water pump. All work in Booster ended and Pbar began stacking at 12:43 PM. MI personnel asked Operations to reduce their stack repetition rate to three seconds to try and keep the LCW heat exchanger outlet temperatures below 100˚ F. Operators reported a power supply (E:V108) that was drifting high and pulling the NuMI beam permit; an EE Support person investigated. FESS personnel fixed the AC for MI-10 at 4:20 PM; the cooler air enabled the BPM crate to resume working.

During the evening shift, at 4:05 PM, the MiniBooNE beam permit tripped off on a power supply (MBEX) drifting low; it reset. FESS personnel installed a portable AC unit in MI-10 at 5:29 PM. At 6:21 PM, a MI kicker (I:KPS6N) tripping off; operators investigated and then called in a Kicker person. Operations disabled beam to T-1008 at 6:36 PM. TEV personnel began an end-of-store crystal collimator study at 7:01 PM. At 7:11 PM, store 8907 aborted due to a TEV RF station (TRF2) tripped off. Operators prepared D0 for access at 7:59 PM Operators tried unsuccessfully to reset I:KPS6N at 8 PM, but at 9 PM it had cooled enough to reset. Operators reported problems with the TEV sector D4 LCW pump readbacks; Water Works will investigate in the morning. The MI Kicker person reported at 9:54 PM that I:KPS6N had tripped off due to high temperature. D0 came out of access at 10:40 PM.

Thursday July 21

The midnight shift began with Operations monitoring the stash (377.5E10), with Pbar stacking (14.2E10), and with MiniBooNE taking beam.

During the midnight shift, operators put the TEV through a dry squeeze at 12:31 AM. At 12:48 AM, Pbar stopped stacking; operators investigated and found that a Pbar Debuncher kicker (D:EKIK) had tripped off. Operators put Pbar into standby and called in a Kicker person. Operations put the TEV into shot setup around 1:06 AM. A Pbar Kicker person reported that he changed out the trigger. Operations established store 8911 at 2:14 AM with average initial luminosity (AIL) of 364.88E30. Pbar resumed stacking at 2:22 AM. Operations established beam to T-1008 at 4:17 AM. Interlock technicians reported that they had completed safety system tests for NuMI at 7:30 AM.

During the day shift, Water Works personnel reported at 8:52 AM that the TEV sector D4 LCW pump was back online. Operators reported at 9:54 AM that the AP2 transfer line BPMs were showing stale data; operators contacted a Pbar person who power cycled the BPM crate at AP-50. NuMI personnel configured the NuMI horn power supply for anti- neutrino operation. FESS personnel reported at 2:04 PM that they had started cleaning the TEV sector A4 pond water strainer.

During the evening shift, Operations disabled beam to T-1008 at 4:52 PM. Operations terminated store 8911 at 4:53 PM, put the TEV into shot setup, and established store 8912 at 6:15 PM with an AIL of 316.16E30. Operators reported at 6:41 PM that they were preparing MI for an access to search for a LCW leak. Also, NuMI personnel will come along on the access to inspect the cables for a magnet (E:V108). While preparing for access, operators found a soft ground fault between MI-30 and MI-50. The Crew Chief turned off the MI LCW at 6:45 PM since the suction and discharge pressures had dropped to about 5 psi. Water Worked personnel put column 1 at CUB into internal recirculation mode at 7:15 PM. Operators reported at 8:40 PM that they found water gushing from a broken ceramic connector. Water works personnel began repairs. The NuMI personnel looking at E:V108 reported a failed sump pump in the NuMI stub; FESS personnel investigated. Water Works personnel reported at 11:04 PM that they had completed their MI leak repair. Operators reported at 11:09 PM that they were cleaning up water in the NuMI stub area. Operators tried to turn on the MI LCW system at 11:57 PM.

Friday July 22

The midnight shift began with Operations monitoring store 8912 (118.76E30) and the Pbar stack (48.1E10), with no stash due to MI work, and with MiniBooNE waiting on beam.

During the midnight shift, at 12:04 AM, operators reported that they could not keep the MI LCW pumps on; a Water Works person investigated. Water Works personnel reported problems with columns 1 and 2 at CUB and a loss of about 1600 gallons of water from the MI LCW system, which they said needed to be filled before trying to turn the MI LCW pumps on. Operators reported at 3:14 PM that the Tevatron Electron Lens (TEL) began tripping off repeatedly; a TEL person said he would investigate in the morning. A Water Works person reported at 4:29 AM that he had all the MI LCW pumps on. Operators reported at 5:15 AM that the A1 transfer line water flow didn’t come back fully; operators checked the A1 service building for a LCW leak, but it was dry. At 5:59 AM, operators reported that during the MI check, the hipotter kept tripping, but they continued with the turn on; they notified a MI person. At 6:16 AM, the MiniBooNE horn would not turn on due to a LCW indication; operators contacted MiniBooNE and safety personnel. Operators began ramping MI at 6:55 AM. Operators informed the Run Coordinator and NuMI personnel about the status of E:V108 at 7:26 AM. At 7:31 AM, operators reported that over the last hour there was a slight loss in luminosity and some beam position problems; operators investigated. Operators reported a problem with a Switchyard quadrupole magnet power supply (S:QP3); an EE Support person investigated. At 7:59 AM, operators received a report that the humidity in MI-12B had shot up at 3:29 AM; operators notified a MiniBooNE person that there was a possible leak in in the tunnel. He wanted the area prepared for an access. Operators reported short down times due to LRF1 & 2 and BRF12 & 13.

The Plans for Friday and the Weekend

The plans for today and the weekend are to stack, stash, and store, but the Run Coordinator wants to try and resolve E:V108 issues, which might require yet another MI access.

Accelerator Update Archive

More Information
For Tevatron luminosity charts and the current status of Fermilab's accelerators and detectors (live!), please go to Fermilab Now.

Comments and Suggestions
What do you think about the Accelerator Updates? Please send comments and suggestions to: accelupdates@fnal.gov.