Accelerator Update for April 11, 2007 - April 13, 2007
Wednesday April 11
The day shift began with the Tevatron (TeV) down due to a power glitch, with Operations monitoring the Recycler stash (156.8E10), with the Antiproton Source (Pbar) stacking (35.1E10), with the Booster, Linac, Main Injector (MI), and Recycler operating normally, with NuMI and MiniBooNE taking beam, and with D-Zero (D0) and the Collider Detector at Fermilab (CDF) waiting for the colliding beams.
At 8:57 AM, the MI beam permit tripped off for the first of about 10 times in 10 minutes due to bad beam positions. It reset.
Operations put the TeV through a dry squeeze at 9:47 AM.
Operations turned off beam to NuMI at 9:49 AM in preparation of their kicker cooling flush.
At 11:32 AM, Operations reported that the gradient voltage for a Linac RF station (LRF2) was jumping around. An expert investigated.
Operations put the TeV into shot setup at 11:37 AM.
Operations turned off Booster and Linac and prepared them for access at 1:01 PM.
Operations turned off MI at 1:40 PM.
Operations reported at 1:46 PM that MI and NuMI were both in access. Linac experts had begun their work on LRF2.
Operations reported at 1:53 PM that experts had begun work on the MI-10 kickers. They completed the work about an hour later.
At 2:37 PM, Operations reported a vacuum leak in the Booster. Vacuum technicians investigated.
Linac techs replaced the Power Amplifier (PA) for LRF2 at 3:09 PM. They reported that it would take another hour to tune the station.
At 3:19 PM, Operations reported that the LRF2 LCW pump was failing. LCW experts began repairs.
Vacuum techs reported at 6:35 PM that they found and replaced a bad O-ring in Booster.
MI came out of access and operators began its turn on at 6:52 PM.
Operations established beam to the Linac absorber at 7:14 PM. An expert began a LRF1 study.
At 8:51 PM, the MI power supply loop beam permit dropped at MI-20 due to an overcurrent indication on a filter choke. Operators reset the permit, but it tripped off again. Operations called in an expert.
A Proton Source expert helped get beam through Booster and on to MiniBooNE at 9:31 PM.
At 9:34 PM, Operations reported that the LRF2 RF LCW pump was leaking about four inches of water per hour. (Operators kept filling this LCW system until morning.)
At 9:36 PM, BRF14 began repeatedly tripping off. Operators bypassed the station and turned on BRF19 in its place.
The MI power supply loop problem was fixed by bypassing the MI-20L power supply. (MI can operate with 11 of its 12 total power supplies.) Operations established beam to NuMI and Pbar at 11:05 PM.
Thursday April 12
The midnight shift began with Operations monitoring store 5348 (55.18E30) and the stash (11.0E10), with Pbar stacking (60.5E10), and with NuMI and MiniBooNE taking beam, but MiniBooNE was suffering from beam position problems.
Recycler studies ended at 12:43 AM.
Operations reported at 1:07 AM that the MiniBooNE beam position problems were due to its autotune program. Operators reloaded the program’s corrector files and beam positions returned to normal.
At 2:48 AM, Operations reported that the Linac beam intensity had dropped due to problems with LRF2. Operators halted Linac beam and investigated.
Operations reported at 3:25 AM that Linac resumed sending beam. They correlated the many problems they were having to the leaky LRF2 water skid. Every time they refilled the system there would be a RF phase jump. On further investigation, operators reported that the I- Source had caused its vacuum gauge to trip off. Operators adjusted the Linac low energy buncher phase and raised the intensity. All of these phase problems were traced back to LRF2.
At 6:02 AM, KRF6 tripped off. It reset.
At 7:57 AM, KRF4 tripped off. It reset.
Operations reported at 8:13 AM that the Recycler had stashed antiprotons three times during midnight shift.
At 8:13 AM, the NuMI beam permit tripped off due to beam position problems. Booster also tripped off at the same time during a beam injection. Everything reset.
Coincidentally at 8:13 AM, operators found that LRF2 was slightly out of tolerance. Operators set it back to normal. This occurred three more times and then operators contacted Linac experts.
At 8:15 AM, the I- Source vacuum gauge tripped off. It reset.
At 8:24 AM, KRF3 tripped off. It reset.
At 9:28 AM, Operations reported that LRF3 had tripped off twice – probably due to LRF2. The Neutron Therapy Facility (NTF), which was preparing to treat a patient, had to stop tuning while operators brought the station back on line and made sure everything was operating correctly.
At 9:46 AM, Linac experts began working on LRF2. LCW techs began repairs on the leaky LRF2 water skid.
A RF expert brought BRF14 back on line at 10:05 AM. Operations turned off BRF19, which was the spare station.
Operations prepared Linac for an access at 10:05 AM.
D0 began a 10-minute luminosity study at 10:16 AM.
LCW techs reported at 10:54 AM that the LRF2 LCW pump they had just installed seized. They needed to replace it again.
Linac came out of access at 11:14 AM.
Linac experts reported at 11:32 AM that they had repaired the LRF2 modulator.
At 11:52 AM, KRF4 tripped off. It reset, but it had no gradient voltage. Experts investigated.
At 12:03 PM, Linac experts reported that got the pump presently installed operating well enough to treat a NTF patient. They will install the rebuilt pump after the treatment ends.
Linac resumed sending beam to MiniBooNE, Pbar, and NuMI at 1:23 PM.
At 1:50 PM, a TeV RF station (TRF7) tripped off. An RF expert investigated.
At 2:04 PM, LRF2 tripped off. The station brought itself back up by 2:14 PM. An expert tuned the station.
NTF reported at 3:03 PM that they had completed treating their only patient for the day.
Linac techs began installing the rebuilt LCW pump for LRF2 at 3:10 PM. They finished about 50 minutes later and there were no leaks
At 4:07 PM, LRF4 tripped off. An expert investigated.
Operations reported at 4:34 PM that LRF2 was up and running.
With LRF2 up, Operations had beam established to MiniBooNE, NuMI, and Pbar by 4:43 PM.
At 5:06 PM, LRF2 tripped off. It reset.
Operations terminated store 5348 at 5:31 PM.
Operations put the TeV into shot setup at 5:47 PM.
At 6:45 PM, Operations reported problems with the TeV tunes that were delaying shot setup.
Operations put the TeV through a dry squeeze at 6:50 PM.
At 7:28 PM, Operations called a TeV expert for help with the tune problems.
Operations reported at 8:03 PM that the tune problems were associated with the new sextupole circuits at sectors A2, C2, and E2 receiving bad values from a recently updated database. Experts investigated.
Operations resumed shot setup at 9:34 PM.
Operations established store 5351 at 11:29 PM with an initial luminosity of 182.3E30.
Operations reported at midnight that the Recycler had stashed twice during the evening shift.
Friday April 13
The midnight shift began with Operations monitoring store 5351 (152.25E30) and the stash (44.3E10), with Pbar stacking (14.9E10), and with NuMI and MiniBooNE taking beam.
At 5:50 AM, KRF6 tripped off. It reset.
At 6:08 AM, ARF1 tripped off. It reset.
At 6:35 AM, Operations reported that the Tevatron earthquake monitor (tilt monitor) registered what turned out to be a 6.3 magnitude earthquake located near Guerrero, Mexico (40 miles northwest of Acapulco) that occurred around 12:42 AM Central Standard Time.
NTF began preparing at 7:42 AM to treat patients later in the morning.
At 7:43 AM, the I- Source sparked and its intensity didn’t recover.
Operations reported at 7:56 AM that the Recycler had stashed three times during the midnight shift.
The Plans for Friday and the Weekend
The plans for today are to stack, stash, and store. The Run Coordinator will allow a MI study.
Comments and Suggestions
|last modified 04/13/2007 email Fermilab|