I forget if I’d mentioned just what fun it was to get this line separated and get the test adapter into place but uhhhhhh, it was a battle.
This thing is pretty cute. There’s a long wire that goes up its hoistway and the control panel in the cab, powered by a rechargeable battery (please don’t forget to plug in the charger!) inductively couples to it to send control tones to the box at the bottom and audio to the cab intercom. Somewhere in this system there’s a 2-way radio, I have no idea where, but the Morse code ID from the news department repeater sometimes blasts out of the intercom at the tower base and spooks the hell out of me because I never expect it.
Alas it doesn’t work because somewhere up the tower, a limit switch circuit broke. I’m not sure where but that’s finally supposed to be fixed this week, after which we can have a tower crew figure out what went arcy sparky 1600′ up. The fault is 1700′ from where the test equipment was connected, but that’s subtracting the 100′ or so horizontal run before it meets the tower and goes up.
So speaking of things blowing up, I love it when people send me pictures of PTek gear, it cracks me up to see, uh….
Not a watt comes from that whole stack. Conversely:
That’s a nice amount of power from an amplifier that’s turned off! Oh hi. #fnord
I forget who made that glorious image but I suspect it’s Duffy Toler.
Anyway, that’s there because it started raining here in Northern California. Duckies would be very happy with this. Quack.
The rain has helped damp down the Mosquito Fire threatening the communities up in the Sierra Nevada foothills. It’s been really nice, even when it got blindingly thick on the Delta, and came down hard enough one morning to make my drive to work slow. I got there late and found the whole facility in headless chicken mode as an unusual combination of issues with how we procedurally handled breaks on streaming only newscasts caused a big ugly failure on air. We’re producing more live programming for television and streaming than ever and the engineering staff keeps getting smaller and more overstressed and something’s gonna break down eventually. Meanwhile, I was exhausted because I’d been down at the transmitter site late in the evening trying to fix the transmitter side of this hot mess, and I experienced a SMOKE TORNADO as one of the power supplies blew up. Impressive. Seriously impressive. Pictured here: the slowly growing heap of transmitter parts that need to be repaired or replaced.
I continue to be approaching the level of burnout this power supply managed. We had a meeting with local management today and got to express our concerns. Will corporate listen and take action so we don’t just wind up in a downward spiral of cascading failures and increasing understaffing? We can’t just hang on like this forever. This blows serious capaci-smoke.
Big thanks to Dan at GatesAir for offering me useful guidance with this transmitter. His biggest piece of useful advice on the ULXT series: once it’s up and running and fully stable, don’t mess with it, just let it run! Well, I guess we broke that rule of guidance. Really the biggest issue is that the antenna system is unhappy SOMEWHERE, I’m waiting on a tech to come by to sweep it and find the distance to the fault.
What follows this is a big ol’ image dump. Click to be, uh, dumped at I guess.
I just got a Flipper Zero and it allows for saving and replaying the magic from infrared remotes. I had a preservation target in mind, just in case –
Primeview monitors… they’re fancy, expensive, have great color rendition and can match your on-set lighting for film and TV very well, and they use an akwardly hard to obtain remote that’s, well, uh, hard to come by and awkward. I dunno. It has a lot of buttons that probably never get used by the Primeview monitor and it looks like it was originally intended to be the generic remote for some other kind of TV with a digital broadcast and radio tuner (!). Anyway, here’s all the buttons, use whichever ones you need. 🙂
I don’t really know what that file format is but it looks fairly straightforward and may be usable for LIRC and other configurable infrared remote control systems.
As I started to get down into the buttons at the bottom of the remote, I noticed the RGB LED light strips on my workbench starting to cycle between colors. Aha, wehave a bizarre match! I remember that the Favorite channel button was one of them.
(I actually fell asleep with my finger on the H key for a moment)
So, three days and a grand total of about 6 hours of sleep into a 5 day work week with ugly overtime that should have been 4, I came in and one of the two transmitters had SHIT ITSELF. It was putting out like 19% normal power. Curious. Logging in and taking a look around, I saw that the B side had cut out and would not come back on willingly. It was time for a quick visit to the site that would turn out to be 12 1/2 hours of grind for no benefit.
Here’s the poor thing in question. Ignore the red toolboxes, they aren’t part of the transmitter, but this is the system. Note the big blue cabinet at right, it’s a Bird “Digital” air load, and it corresponds to the resistor logo shown between the A side and B side in the diagram above. It’s the main reject load that will dump any power resulting from an imbalance in the system. The combiner is the big black chungus (it’s kinda X shaped if viewed from below) above the right side of the transmitter cabinets. The black object above the Bird load is the antenna switch, corresponding to the “baseball” switch on the right side of the diagram – it lets you select between the main liquid cooled test load (not seen in the picture, it really doesn’t look like much either – just a weird canister with two big hoses) and the antenna system.
This is the architecture. The temporary line that bridged the A side to the Space Station Toilet is seen near the top of the picture. It was used when the Space Station Toilet was replaced with the Great Wall of RF or whatever the hell that thing is. It looks like the above transmitter but taller and newer. I dunno. It’s a combined bunch of ULXTE cabinets and Just Works.
So, the reason this temp line came into play and I noted it here is that this transmitter had to play double duty for a little while. It’s basically two independent ULXT-40 transmitters combined with this MSC2 interface in the middle cabinet to be one bigger transmitter. While the Space Station Toilet was being removed and replaced, the A side was its stand-in. The B side remained in place and I guess had been set up to bypass the X shaped hybrid combiner. The vertical rectangles before the filter are the boxy hybrid couplers (?) seen above the cabinets in the photo and the resistor symbols correspond to the big tall vertical Bird loads on the shelf. The resistors here are big enough to definitely not be futile.
Later on, it was restored to its combined state, and it came back on and started making the Bird aircooled load go start roaring away, battling the air conditioning.
ORIGINALLY, its fans barely had to run at all. And finally, the transmitter noped out.
The original cause was the B side tripping out on high VSWR. I was suspicious of why that happened but the A side did not report unusual reflected power. I got it running again at 100% power, but I noticed about 3KW being dumped into the big blue Bird load. When either side is off, approximately 50% of the running cabinet’s output gets dumped in the Bird.
I switched to the main water cooled dummy load after turning up the aux site, and was trying to get both A and B to run together into the load without much luck. After an initial call to GatesAir support, I pretty much just had a recommendation to power cycle everything. After power cycling everything…. the A side came back with no web interface!!! It just threw a generic 404 error. Now the central controller thing couldn’t see it anymore either, probably due to missing software. It still booted up enough that I had a web server, just with no content. I was able to telnet in and get some screens of readings and commands, but that was it.
At this point I had the support person very stumped…. but finally I remembered there was a much less documented interface on that server at the path /isp/. It let me flash the firmware again, and now I had a web interface in the normal location. Weird, right? More like HHHHHHHHHHHHHHHHHH—––
Now I could try to turn BOTH on from that DualTran interface! Perfect. Oh wait, they both fall on their faces with their ON / OFF lights wigwagging in unison like a railroad crossing, three retries then cutting off. After some fighting with it I got both to work at reduced power, screaming of excessive dissipation in the reject loads pretty much all the way back through the system.
I was on the phone with GatesAir support and the support person told me to just not pay attention to the reject power going to the Bird. Oh, sure—
….say, this looks like a phasing issue, right?
……..boy, that required some pretty coarse adjustments to do it, but the reject power is peaked towards zero…
…………did anyone before me ever bother to check the phasing???
WHO HURT YOU? Geeeez. After over an hour of rephasing things via a really wonky slow interface, I had it looking REALLY good. It was gleefully yeeting 60 kilowatts of heat out into the evening sky via the outdoor heat exchangers.
Also at some point I was stuck for hours trying to figure out why I was getting an “EXT FAILURE” interlock stuck on one, then both cabinets, and eventually tracing it back to invalid configuration settings on the MSC2 – they corresponded to a combiner with phase shifters, which the weird X shaped thing doesn’t have, so of course there were no interlock switches connected to the combiner interlock as there was nothing to protect with an interlock. It was a Fnord Failure so you just can’t see it, and I kept trying to figure out if it was the interlock on the baseball switch…
also I kept suspecting some of this stuff was loose but it wasn’t.
Time to bring the signal home! I switched back to antenna. Now it would only run one side at a time, giving a lousy 25% or so of normal power. Trying to make some sense of it at all, I reduced power and got both to run…. only to be greeted by seeing I was sending 20 kilowatts up, and getting 4.8 back!
Switched to the load, and got 60K forward / “35 watts” reflect (which seems to be the zero point).
WELP. Now it’s time to have the line and antenna sweep tested, I think…
What blew up? And what blew up FIRST? I dunno.
I’m extremely burned out now and just kinda glad it’s not in my hands anymore.
Oh hi. There’s a slightly modified screenshot of a page from our building’s Johnson Controls Metasys UI while it was trying to roast our morning show hosts and stage guys to death and I was trying to get it to kindly, you know, not.
This is for the studio air handing unit. It just blasts cold air into the studio and does not have heating features built in. Various people have asked me at times why the heating doesn’t work. It just now occurs to me I could show them this graphic – no heating coils, reheat, or connection to the gas furnace next to the air handler exist for a now obsolete reason: this studio used to originally be lit by a multitude of terrifying hot lights to the point that heating would not be necessary while the studio is in use. Here’s the air handler viewed from outside:
A couple years ago I used to have trouble with the programming on this drive, but that’s long since resolved. The issue I was having was that it was just constantly getting a command to run at 00.00 Hz, which meant no blowy. A quick glance at Metasys showed me that it was calling for 100% chilled water flow to the coil and it was nice and cold inside the air handler, so I just hit battle short on the bypass there to get it going and cool the studio while I investigated. After the fan came up at 60.00 Hz (absolutely fearsome) it sucked one of the doors closed as I walked past it and tore my pants halfway down the side. Nice.
Anyway… Back to looking at the controls. The system cycled once like that, reaching the setpoint temperature and overshooting it slightly, then slamming shut the chilled water valve and letting the studio overheat again. When it had actually cooled the place down I went back up the roof and took the drive out of override, but now it stopped the fans too. Hmm.
This time I set Metasys in override. On each of those control parameters (return fan output, supply fan output, mixed air damper, and chilled water valve output) I could override them to nonzero values and the command would take effect! The only thing that was missing was any semblance of thermostatic control whatsoever. The system was just any degree between on or off I wanted… but once set like that, the studio temp would just drift up or down… I just about froze Tina and Courtney who were sitting right under vents. Aaaaaaaaaa! A call was placed at 8 am for the one and only controls tech in the area, who came out and spent several hours on the hellbeast.
After a while of scratching his head over the way our BACNET system was configured and how it was just spouting communication errors, the controls tech spotted one of these thermostats on the wall of the mechanical room with its display dark. Curious, we thought… I grabbed a meter and tested between the wires in it and found about 3.6 volts AC where there should be 24.
Sorry, I didn’t get a picture of the inside of it but it’s basically just like any normal thermostat, just with a BACNET connection in addition to the normal R, RC, C, Y, G, W, and Y wires. It basically just lets Metasys schedule occupancy and setpoints and monitor the status and temp. The menu system is maddening.
So I walked over to the electrical panel for the air handler it controlled and saw the breaker appeared to be on. I turned the breaker off and back on and the thermostat lit up and the blower started. 26.1 V AC appeared between R and C.
At this point I went back into the studio, wound up the timer switch on the wall that serves as the manual zone occupancy override, and after a couple minutes, the blowers started and the water valve opened until the discharge air temp met the setpoint. (Yes… It’s even adjustable!)
So uhhhhh, what the heck? My only thought is that when that wall thermostat was getting 3.6v (why that and not zero???) it was just getting enough power to make the microcontroller or RS-485 line driver in there wake up and periodically yeet glitches into the BACNET void
The controls tech explained to me that it works like Token Ring. If the token falls out of a powered down thermostat in an empty mech room, does it make a sound? Really this all reminds me of something the Space Toilet used to do wherein you could mess up its canbus network and make the amplifiers blink on and off like holiday lights from hell.