Kyle Brady: Blog » Security

MediaTemple’s Continued Inadequacy Issues [Expose]

Kyle Brady — Thu, 26 Nov 2009 23:36:02 +0000

[note: substantial and important updates available]

Almost a month ago, I made MediaTemple, and the world, aware of an attack that seemed to be a large security issue, and they eventually admitted it was their problem to deal with, rather than blaming it on software like hosting companies like to do. But, weeks later, the problem is not yet resolved, and the public is largely still in the dark.

In the last week, I’ve been notified twice to change my FTP/SSH passwords, and the request yesterday came with an odd statement: the passwords had been previously stored as plaintext, rather than being encrypted or hashed, and that the attackers somehow had access to this - this was MediaTemple’s sole explanation of the massive security issue.

Entirely unacceptable.

After initially making this issue public, both here and at the Inquisitr, I received a phone call from Andrew Won (VP of Customer Service) and Chris, whose position I can’t remember, on 11/16/2009 saying that they discovered the issue, had patched the necessary software, and had submitted patches to the software’s vendors – but asked me to not say anything because of the “security process”. They didn’t give me enough details as to what was actually happening for me to matter, but I kept quiet.

The traffic on my blog, and the comments, continued to mount in the days that followed and it became clear that the issue had not been resolved – people were still being hit with this hack/attack. On 11/19/2009 I asked for an update from Andrew and received a reply stating:

Unfortunately, we still don't have anything public yet. We have already resolved all of the issues and this issue will not recur.

Well, as the attacks continued for other people’s accounts, even through today (11/26/2009), it’s obvious that they had not resolved the issue. When I irately called late last night (11/26/2009 early morning PST), the tech had no answers and neither did his supervisor – in fact, I knew more about the situation than they did, and I was given the partyline: “our engineers are aware of the issue and working to address it.” Further conversation with Andrew, via email, resulted in nothing but doublespeak and sidestepping my questions.

It’s obvious, at this point, that they are either incompetent or lazy – I’m not sure which. They were slow to respond to this in the first place, and have made one misstep after another, which isn’t giving the affected customers much faith in their hosting company, let alone those unaffected that hear the horror stories. The fact that passwords were stored in such an insecure way might be part of the issue, but there are larger problems: discovery, point of entry, depth of access, and execution – none of which (mt) is, in any way, addressing.

When I mentioned this to Andrew, he responded by effectively saying they still have no idea what the problem is or how to fix it:

We are still in the process of investigating this. Unfortunately, while we have a lot of theories and assumptions, we still do not have anything definitive. So please bear with us while we investigate this. We are taking all precautionary measures and locking down many external and internal systems. We will continue to closely monitor our systems and take appropriate actions.

And they even want to dispute the fact that it’s been almost a month, while downplaying the large number of customers affected:

It was not a matter of resolving over the period of 3 weeks. It was a matter of continuing to take steps, monitor and then take further steps. The number of sites actually affected is very small, but due to recent events, we decided that we needed to take a more blanket security approach and change all (gs) Grid Service Server Admin passwords as a precautionary measure.

The “security protocol”, mentioned above, is essentially a “don’t talk about it until it’s fixed” process, but it assumes that those involved are actually trying to fix it, and (mt) is using this as both a crutch and deflector shield – in addition to assuming unaware customers are happier than aware ones:

Chris and I advised you of security protocol, which is what we were following. And security protocol states that you do not publish public info until you are absolutely certain that the issue is resolved and that you are reasonably certain that the attacks or hacks have stopped.

We didn't have much choice in this matter. As we explained to you before, security is a very sensitive issue and by making information public, you are also feeding information to your attackers. We also alerted all affected sites and accounts of the issue and informed them of the steps that we have taken at the moment and time. This issue was still evolving when we last spoke.

Finally, when asked about compensation to customers for their utter failure as a semi-secure hosting company, which they haven’t actually fixed yet, Andrew once again sidesteps the issue by choosing to blame the users/customers instead of themselves:

We do encrypt passwords, but there was a separate file that was kept for the purpose of allowing customers to view their FTP and mySQL passwords through their Account Center. This was a feature many customers asked for in the past. However, we have decided that this feature comes at a price and we are no longer willing to take that risk. Yes, we have learned our lesson. We definitely do understand that this was a headache for ours customers, it was a huge one for us, so we can only imagine it was a much bigger for our customers. We will make sure to discuss a concession of some sort for those customers that were actually affected by this issue.

In summary:

these attacks are the result of MediaTemple’s failure as a hosting company

they chose to wait three weeks to even address the issue publicly

they claimed to have solved the issue long ago, when they hadn’t

they still haven’t solved the security issue, three-or-more weeks on

they continue to not reveal any details to users, while sidestepping most questions

they seem to have no idea of what is truly occurring

They’re going to lose alot of customers over this, especially since they are known for having large-scale problems on a regular basis.

--- --- ---

Update (11/26/2009 5:30pm PST): I had a lengthy phone conversation with Andrew, and while I can't comment on the details, I feel more confident in MediaTemple's abilities and in what they're doing to solve this large security issue. More concrete details as they come, but I would suggest that we have more patience with (mt) on this.

Update (11/30/2009 4:35pm PST): MediaTemple is slowly opening up about this, although the full story doesn't seem to be public yet. Details as/if they come.

“The Epic Wordpress + MediaTemple Failure” [Self]

Kyle Brady — Sun, 15 Nov 2009 21:57:46 +0000

New column at the Inquisitr:

If there’s a security issue floating around, you’d imagine that those behind the problem would be extremely interested in fixing it as soon as possible… right? Well, apparently not.

Go check it out.

SJSU Mass Email Failure [Expose]

Kyle Brady — Fri, 21 Aug 2009 07:13:41 +0000

Alternative title: "How to Publicize 17,000 Private Email Addresses"

Earlier today, I received an email from "Tameka N. Harris" regarding parking permits at San Jose State University for the upcoming semester. It wouldn't have been an interesting email except for a minor detail:

There were 400 email addresses in the "To:" field, including mine.

Note that it was an actual email, instead of an anonymized message through the PeopleSoft-based system that hides any and all email addresses, usually used for such mass-communication.

After speaking with a friend, who had 700 separate and unique email addresses on his receipt, I discovered it's likely this was a mass email to the entire student body - over 17,000 people, all with email addresses exposed to each other.

I responded to the original, 400-person email in "Reply All" fashion, saying:

I would just like to point out to everyone on this list that Tameka N. Harris, the Almighty Beloved of SJSU Parking Services, has exposed your email address to the world, along with hundreds others, all because she couldn't figure out how to use the "BCC:" email property instead of "TO:".

That's how I'm able to email all of you.

Way to go, Tameka.

I intended, and tried, to email those on my friend's list, but Google prevented me from doing so, and accused me of spamming. Fair enough.

It's important to note just how unacceptable such a huge breach of student privacy this is, not to mention the gross administrative ignorance by both Tameka and SJSU - has she never used email before? "Irresponsible, outrageous, and unintelligent" only begins to describe the situation.

Her original email, which wasn't worth such a mass-mailing, is unedited as follows:

(note: the email has more color and formatting than Wordpress allowed me to copy-and-paste)

Please conserve: Think before you print this e-mail.

IMPORTANT PARKING NOTICE
BEWARE: Limited Parking and Heavy Traffic
During the first few weeks of instruction, traffic is unusually heavy and finding parking is difficult! Please plan accordingly and consider using SJSU Park & Ride or your VTA EcoPass for public transportation. Throughout the semester, the parking garages usually fill to capacity prior to 9:00 am and remain full past noon. UPD Officers provide traffic control during the beginning of each semester. It is important for the safety of everyone that you follow their directions!

THERE IS NO GRACE PERIOD
A valid parking permit is required at all times, including the first day of classes. Parking rules are enforced 24 hours a day/7 days a week. Possession of a permit does not guarantee a space in the main campus garages. Space is always available at the “Park & Ride Lot”. There is NO free parking on the Main Campus.

Avoid traffic and parking frustration…
Use SJSU PARK & RIDE LOT!
Only the “Park & Ride Lot” offers free parking the beginning of each semester (August 24- Sept 3, 2009).
Free parking is only available at the “Park & Ride Lot” August 24 – September 3, 2009. The “Park & Ride Lot” is located 8 blocks south of the main campus on South 7th Street at Humboldt Street across from Spartan Stadium. The parking rate is $4.00 per day (or $96.00 for a semester Park &
Ride Permit). ALL SJSU permits are valid in the “Park & Ride Lot”.
Free Park & Ride Shuttle service is available to campus Monday through Thursday. Free Shuttle Service begins at 6:30 am and runs every 10 minutes (depending on traffic) until 4:00 pm with stops at Park & Ride, Duncan Hall, MLK Library, Engineering Building and Business Tower. After 4:00 pm, Free Shuttle Service runs every 20 minutes until 10:20 pm with stops at Duncan Hall and Park & Ride only. Shuttle Service is not available Friday through Sunday.
All Park & Ride Permits are valid in all student areas on the main campus Friday through Sunday only.

DON’T WAIT IN LINE!
Buy Parking Permits or Pay Citations ON–LINE!
www.sjsu.edu/parking

.. No Additional Fees
.. No Lines – No Waiting
.. Print a temporary permit valid for 10 days
.. Fast delivery to your home without shipping and handling fees
A limited supply of Student Semester permits are available at the Student Services Center – Bursar’s Office located on the ground level of the North Garage(South 9th and East San Fernando Streets) (Cash or check ONLY). Please expect long waiting times during the first few weeks of school.

Student 1-day–a–week, 2-day-a-week and Park & Ride permits are available ONLY at the Parking Services’ Office located in the University Police Department (UPD) building at the South Garage (S. 7th and E. San Salvador Sts.) (Cash or Check ONLY)
Daily Permits: Pay stations are available on the 3rd floor and above in the North and South Garages and the 1st to 4th floors of the West Garage (E. San Salvador and S. 4th Streets).
Daily Rates:
Each ½ hour $1
Maximum Daily Rate $8
Maximum after 5:30 pm $5
Overnight parking $10 (Expires 8am next day)
THERE IS NO GRACE PERIOD
A permit is required at all times including the first day of classes. Parking rules are enforced 24 hours a day/7 days a week. Possession of a permit does not guarantee a space in the main campus garages. Space is always available at the Park & Ride Lot.
For more information or to review the Parking Rules and Regulations, visit our website: www.sjsu.edu/parking
(408) 924–6556
The latest SJSU Safety 101 Uniform Campus Crime and Security Report is available on–line at www.sjsu.edu/safetyreport. A pamphlet can be obtained at the University Police Department (call 408 924–2172 or visit the UPD web site at www.sjsu.edu/police for more information.

This issue follows on the heels of the "Beeson Debacle" from two and a half months ago (also at SJSU), which is about to be revived in the first issue of the Spartan Daily on Monday - 8/24/2009 - the first day of the Fall Semester.

Good timing, Tameka.

The New Era of Spacecleanup? [Old Content]

Kyle Brady — Fri, 13 Feb 2009 14:44:09 +0000

When you're a little, wideyed kid, you hear about space. You see pictures of the Earth from orbit. You see pictures from, and of, the Moon, and you're enamored. For some, life goes on, but for others space remains a thought in the back of your mind to one day show itself via a career in engineering or physics.

I'm somewhere between the two.

But what they don't tell you as a little kid is how cluttered both Low and High Earth Orbits are becoming. As we've seen, collisions in space are far too real, and the "space trash" problem is apparently becoming worse, as more and more people send up devices, sometimes only for testing purposes, that largely get left when their creators are done with them.

The official story is that the American satellite was a Motorola one, but I'm betting that's, at best, only part of the truth, since any number of government organizations have the option to piggyback on your space hardware. Put this together with the fact that the Russian satellite's collision came as a surprise, when we supposedly track "all objects larger than a football" in orbit...

And I think you have a recipe for a new era of spacecleanup. Maybe "cleanup" isn't the right word, but at a time when satellites are destroying each other and being shot down from Earth, I think our military agencies are going to suddenly care more about the spacetrash orbiting our planet.

The idea of an automated, or semiautomated, orbit debris cleanup system via robots with manuvering capabilities is not new. But the realization that we're fixing our own satellites, and, in all reality, disabling others, is new. If we can manage to fix and destroy satellites from orbit, then wouldn't creating a dumptruck-like manuverable robot be less difficult?

I can imagine this trash collector orbiting Earth, selecting which debris is trash (based on human-maintained lists), and scooping up the true trash. After compacting it, ala Wall-e, it could be sent in a proper trajectory to burn up in the atmosphere.

--- --- ---

Further Information: For some pictures and video check out the coverage on the Inquisitr [1] [2]

Facebook Re-Design FAIL [Expose]

Kyle Brady — Sat, 26 Jul 2008 21:02:07 +0000

If you haven't heard or noticed by now, Facebook is in the process (or is finished?) rolling out a redesign of ... pretty much everything. Check out www.new.facebook.com to activate it. But I'm not here to analyze it like everyone else, even though I think it's a pretty cool update.

I'm here because they screwed up.

I've been keeping an eye on the "applications" page, because I had a feeling that something was going to happen... and it did.

Click, because it's relevant.

See all the things I've circled in red? Most of those are applications I've never added. Ever. And in the case of "Bumper Sticker", "friendbinder", and "Top Friends"... those are ones that were added and removed (before the design change) within a 24 hour period.

My first question is: what the hell?

Is Facebook just randomly letting applications access my data, and decide that I'm now a "user" of them? Because, if so, that's not only stupid and wasting my time... but it's also a huge privacy concern.

My second question is: what the hell?

I've tried to remove ALL of the ones that I circled in the screenshot... they won't go away. They disappear from my profile, and from some of the settings pages... but remain on others. Which would lead me to believe they're not really gone, they're just pretending to be.

My third question is: what the hell?

There have long been rumors that Facebook doesn't actually delete any data, they just "delete" it. Instead of removal, a field is changed to tell the rendering engine "Hey! Don't show this!" - which might make sense in some cases, but not as an overall policy.

This is solid proof that they do exactly what people have been whispering about... besides the whole "delete my account" controversy, of course.

My fourth question is: what the hell?

I'm actually out of "Items for Hell" at this point.

Your Mission

This needs to be fixed immediately. Check your applications page, see if you've got anything weird going on. Send them feedback (using the "send feedback" button... obviously), regardless of whether or not you have this problem... they need to know that many people care about this, and it's kind of a big deal.

p.s. Yes, I'm still using Vista on this computer. But that's because I haven't gotten Ubuntu running yet... the RAID-1 array and the supersexy, but "too new", combo optical drive are creating major problems.

--------------

Update (7/28/2008 2:30pm PST): SitePoint blogger Josh Catone (formerly of RW/W) picked this up, and wrote his own take on it.