Category Archives: What have I discovered?

Not OK, Google

Last weekend I logged into my credit card company web site and accessed the page of my transactions.
Suddenly something strange caught my attention, my Google Chrome browser presented the icon of “unsafe site”…

In my credit card’s web site?!
Not on my watch… 😉, so I looked into the page’s source code and Chrome’s warnings, were one main finding stood out – a red colored error stating:
“Mixed Content: The page at ‘https://<site-domain-name>/Card-Holders/Screens/Transactions/Transactions.aspx’ was loaded over HTTPS, but requested an insecure script ‘http://www.gstatic.com/charts/loader.js‘. This request has been blocked; the content must be served over HTTPS.”
(You can test it yourself using this site, which will show you the server’s response)

Hmmm… who owns gstatic.com? well, Google! The same one who develops the browser that I am using, Chrome… ahhh, the irony….

And as far as I know, Google really works hard to apply strong security and SSL/TLS everywhere, at its sites and services, and pushes the whole Internet towards security. Even the Google’s “unsafe sites” help page states that missing SSL (“secure”) is one of the reasons to present the “unsafe site” icon:
“This page is trying to load scripts from unauthenticated sources: The site that you are trying to visit isn’t secure.”

First thing first – I informed this issue to the relevant person at the credit card company, to simply change the URL to use HTTPS.

Then, being the good digital citizen I am, I turned to Google’s security team and filed an issue about the above.

At the same day they replied:

Status: Won’t Fix (Infeasible)
Hey,

Thanks for the bug report. We analyzed it, but there are still some areas we don’t understand fully.

How could this be used in the attack against other users? Please write a more detailed attack scenario – we have prepared some tips on how to create one at https://sites.google.com/site/bughunteruniversity/improve/writing-the-perfect-attack-scenario.

Thanks a lot in advance!

Regards,
<name>, Google Security Team

I didn’t even bother to write such a report. SSL/TLS is so basic, that we shouldn’t even drill into reasoning it these days.

In addition, while writing this post I also noticed that http://www.googleadservices.com is also allowing plain text access and does not automatically redirect to HTTPS.
It is also owned by Google and you can also see its server’s response.

Since I know Google is VERY security oriented, this looks strange to me as something that was possibly overlooked. Is it possible that it is so on purpose? To allow access by weaker/older clients? other reason(s)?

 

Update, 13-Nov-19:

Google team just updated their response with the following text:


Migrating all the domains to HTTPS, and deprecating all clients that can only talk HTTP takes time. We’re constantly trying to add HSTS support to various services, but we know there’s still much to do in this area. As we already know about our HSTS posture and are actively working on this, we don’t treat the lack of HSTS for a given domain as a bug that needs a separate response, and tracking (see https://sites.google.com/site/bughunteruniversity/nonvuln/lack-of-hsts). Thanks for your research and better luck next time!

It didn’t really changed my mind. I am sure they know they can at least auto-redirect the clients, using the server’s response, to the HTTPS version of their sites. My guess is that this is done to try and reach more clients for their services, probably mostly their ads service.

gstatic.com was registered on 2008. googleadservices.com was registered on 2003. We’re towards the end of 2019. They didn’t have the time till now to add SSL to these sites? The mighty Google?
The one which gives better search engine rank to sites with SSL (from 2014)? The company which runs “Project Zero” (also from 2014), which aims to find vulnerabilities at non-Google services and products?

Before you educate the world – clean up your own stuff. Be the example to follow.

It is not a vulnerability. It is a feature. A Zendesk customer? Act now!

I am not a Zendesk expert but I have seen enough. Here is my story.

The short version:

If in your ZD settings the check box of “Require authentication to download” (in the site path of Admin > Settings > Tickets) is NOT selected (hence Disabled) – there are web address/URLs at sub-domain sites of zdusercontent.com that store files that are accessible without any authentication, anonymously, and they can be files that hold private data of your customers.

This check box is disabled by default! By a ZD intentional decision. Customers of ZD may not even know that and not change this default and thus the files will remain accessible anonymously. Not nice. I guess not GDPR compliant.

ZD support article about this check box – https://support.zendesk.com/hc/en-us/articles/203927716-Attachments-in-Zendesk-Support#sec5

Some of this data is indexed by Google, sample searches:

site:zdusercontent.com

https://www.google.co.il/search?q=site%3Azdusercontent.com

site:zdusercontent.com receipt

https://www.google.co.il/search?&q=site%3Azdusercontent.com+receipt

And so on – try the words like bank , “credit card”

Also, if you go to the following link you can find who ZD customers are

https://www.zendesk.com/why-zendesk/customers/

And then you can search by their names, Say, Uber

site:zdusercontent.com uber

https://www.google.co.il/search?q=site:zdusercontent.com+uber

These URLs are quite long and use complex and random characters, so they are not easy at all to guess. But, they can be sent to your customers from the ZD system as links in emails (which can be exposed in many ways) or they can be logged in your security systems, hence exposed to your IT team (see the longer version of this story below).

Since these URLs are accessed anonymously, I guess the only possible way to track who used them is by source IP, which of course can easily not be the real IP of the person who initiated this access (say if the person is using a proxy server or public VPN service).

So, my recommendation to you is to enable this check box, which will change this behavior and any access to any attachment file will force the accessing person to first be authenticated by the ZD system.

This may have negative operational results for the ease of your customers’ access to these files – so weight the pros and cons before doing this change.

The long story:

One day last week, as I was reviewing our gateway alerts, I noticed a strange link, beginning with a sub-domain of zdusercontent.com and followed by medium size string of a URL parameter. I searched to find who is the owner of this domain and found it is owned by ZD. Cool. Safe. I clicked it.

A JPG file loaded into my browser. It was a photo a customer of ours took, to prove his identity, a personal identification document… whoaaa… what???

Although I knew I didn’t log into ZD recently, I cleared my browser’s cache and all cookies, and tried again. The same…

I tried using another browser. The same.

I tried from another PC inside the company. The same.

I tried from my private mobile phone, I tried from my home. All the same.

I tried another link found for this domain – a zip file with multiple files sent by another customer. Not nice, not at all.

Woooo, I said to myself, we’re going to make tons of money on this one via a bug bounty. Zero authentication for customers’ private data. No joke.

So, I found ZD bug bounty page at HackerOne – https://hackerone.com/zendesk

It didn’t mention that the domain of zdusercontent.com is included in the bounty program.

I didn’t give up – I asked HackerOne about it, but quickly I learned HO is not really responsive nor knowledgeable so I turned directly to ZD security.

They promised me that zdusercontent.com is included in the bounty and that they wish to accept my report. (BTW, even now this domain is not mentioned as eligible to their above bug bounty page).

So I PGPed my findings and sent it to ZD, including an offer to simply block search engines from indexing these sites with a simple robots.txt file.

They replied:

To summarize the issue you reported to us, you found files (Zendesk ticket attachments) were indexed by the Google search engine and could be accessed publicly, without authentication, and in some instances without the token parameter in the URL. If I have missed anything please correct me.

This specific topic is something which has been brought to our attention before and has been discussed internally. I want to assure you everything is currently working as intended. All Zendesk accounts have an admin setting to require authentication to view/download a ticket attachment: https://support.zendesk.com/hc/en-us/articles/203927716-Attachments-in-Zendesk-Support#sec5. If you are worried about potential information disclosure please enable that setting to restrict access to all ticket attachments, including the files which are indexed by search engines. In that page you can see the only time ticket attachments are indexed by search engines is when the links are posted on third-party public websites. The files being indexed are not being leaked from Zendesk, but intentionally posted to public locations. This setting exists because the feature of publicly shareable ticket attachments are a popular request from our customers. That being said, I completely agree with you that there is no reason to not include a robots.txt file for that domain. There is currently an open request to implement robots.txt on a few domains which handle customer attachments which should be rolled out by the end of the year, if not much sooner.

In the meantime, if enabling the “require authentication” for attachments doesn’t fit your organization’s needs, please take a look at our Attachments API which would allow you to handle attachments on an individual basis. You can redact comment attachments via the information provided here: https://developer.zendesk.com/rest_api/docs/core/attachments#redact-comment-attachment. You can permanently delete uploads via there information provided here: https://developer.zendesk.com/rest_api/docs/core/attachments#delete-upload.

And their next response after I replied to the above with amazement to their answer and asking if this check box is disabled by default:

The administrative setting of “Require authentication to download” is disabled by default. Many of our customers specifically ask for the ability to host and share non-sensitive documents with their customers so we give them the ability to configure their account to best fit their needs. Additionally, many of our customers’ customers utilize Zendesk strictly through e-mail and not the actual Zendesk UI, therefore they wouldn’t have a registered account to begin with which could cause a lot of confusion if the e-mail correspondence contains attachments. That being said, I’ve escalated this to my manager to start the discussion with our Product teams regarding the default nature of that setting. I can’t promise a change as this may be an accepted risk using the shared responsibility model.

I’ve already mentioned the API endpoints available which Zendesk accounts can use to redact/delete all non-inline attachments. If you are worried about any specific attachments I would reach out to that specific account and address the issue with them. If they are unsure how to proceed with that type of request Zendesk is more than happy to walk them through that process.

I think ZD is making a mistake by disabling this check box by default, loading on itself the legal responsibility for data exposure, when some of it may be private.

If they enabled it by default – they would have been covered themselves better, making the default more secure and if customers would try to change it – they would be displayed with a flashing bold warning about the consequences of such change, and if they do change it – they will be the ones responsible for any relevant data exposure.

So, there goes my imaginary pile of bug bounty money, but as least I came across a good story and a chance to let you know about this risk and possibly mitigate it.

A next day addition I forgot to add – I found only one place on the web that already related to this issue – and it is from a poker forum, where a customer complaints about his personal data being freely exposed on the web, in a zdusercontent.com sub-domain. And he is furious… – https://forumserver.twoplustwo.com/252/global-poker/security-issue-personal-documents-posted-open-web-1715425/

(4-May-2020 – I added here the above forum discussion as a PDF file, for archiving – zd-angry-customer)

Addition at 22-Nov-18: Hi Zendesk folks, I see you reacted quickly and cleared the search results from Google. That’s very good. Just remember there are more search engines you need to handle, some main ones:

Bing – https://www.bing.com/search?q=site%3Azdusercontent.com

Yahoo – https://search.yahoo.com/search;?fr2=sb-top-search&p=site%3Azdusercontent.com

DuckDuckGo – https://duckduckgo.com/?q=site%3Azdusercontent.com&t=h_&ia=web

And I’m sure you will find more.

“VirusTotal​ Windows Uploader” poor design of privacy

Something to share with you, which I am not sure is known enough:

Recently, while I was tweaking a network monitoring systems, I noticed an upload of a file that its name included a full local Windows file path, ending with a name of a file I uploaded to VirusTotal, using their Windows application – “VirusTotal Windows Uploader“, version 2.2, which is the most recent version.

Looking deeper into this I found that uploading a file using this app is performed in a way that:
1. The upload is performed via plain text HTTP. No SSL/TLS based HTTPS is used. Just for comparison – the web site of VT, and its API, forces the use of HTTPS to upload files
2. The uploaded file name is not merely the file’s name and extension – but rather the full path of the file, from the drive letter up to the extension, like “c:\users\dan\Downloads\file-name.exe”

Neither of these issue can be change by the user of the app. The app’s interface doesn’t have any options to change these issues.

Attached at the bottom of this post – a screen shot of a network packet capture that I made – demonstrating these issues.

I realize this app is rather old, possibly from 2013 by its attributes, but I was not expecting that either VirusTotal or its parent company, Google​, who both care about information security – to have such a weak privacy design, running around for so many year, without even informing the users of this app about this way of work, in the app’s page (in the link from above).

I approached VT about these issues, by email, and I got this response:

We haven’t updated the uploader in some time, so there are certain issues like that, and we can take them into account. In the meantime, you are welcome to use the Public API to build an uploader setup that you are more comfortable with.

I hope that VT will, ASAP:
1. Use the app’s page on their site to inform users about these issues
2. Create a new version of this app – one that use HTTPS, possibly using their own API, and of course – upload only the core name of the file, not including its full path as part of the file’s name

FYI.

Eitan Caspi