Thursday, May 25, 2017

bad request 'limit request headers fields size' opendx microsoft

Some Microsoft sites returns the following HTTP error in Google Chrome only 


Bad Request 

Error parsing headers: 'limit request headers fields size'


Microsoft's OpenDx  learning site returns a Bad Request (May 4th-6th, 2017, May 25, 2017) in Chrome.

https://openedx.microsoft.com/


This is new Microsoft Learning site hosted on Azure (Microsoft's cloud service) using Open edX an open source training platform from Harvard and MIT. 


Explanation



This is caused by a browser initial accessing the page on the server. The server is asking for information in an HTTP request from the browser, usually this is an cookie that in part contains your authorization information. Each time you access the site the cookie is updated and depending on its design, it can grow. If it grows two large then you can get a Bad Request error from the server. If you get this error, then the new cookie information is never set for this domain.


This can also be caused by changes to code in the page setting a new cookie definition, or a change to web server configuration. If web server is being rebuilt nightly, this this could cause this affect the cookie definition, invalidating previous cookies.


Solution


Clear your Chrome cookies with a free tool (see my post on this) for opendx.microsoft.com to verify and delete them. You do not have to clear all your cookies for all sites. 

It seems for the openedx.microsoft.com Chrome is not expiring its cookies properly. 


But the error may persist, depend where you are on the build cycle of this site. 



I will demonstrate that this is not a cookie issue for me. On May 4th, 2017 and May 25, 2017 https://openedx.microsoft.com/ had this error in Chrome. 


I cleared my cookies for opendx.microsoft.com and still I could not get into the site. 

I revisited the page and check the cookies for opendx.microsoft.com and Chrome doesn't even have a chance to set a cookie. 

However in Mozilla Firefox, for the site https://openedx.microsoft.com/ sets a csrftoken cookie, but not in Chrome. 

In this instance, it probably is a server configuration error. Following that lead, I found 
openedx.microsoft.com is running on Nginx server.  Read my post on how to get the server manufacturer using curl for windows, to check this issue for other servers.

Nginx servers have a small default header size, see below. 



In the end, this error will be resolved quickly, usually in the same day, because the site cannot negate Chrome representing 60% of market share for long.


But in the meantime, use Microsoft IE or Mozilla Firefox browser instead, until this is server configuration is fixed.


NGIX Server Hosted Websites cookie issue 


Lately the Bad Request error is occurring many times on Microsoft and Microsoft Affiliated Websites when using Chrome.

One began to suspect that this was a ploy by Microsoft to generate traffic for IE. However, most likely its just poor developer testing.

For sites hosted on NGIX servers, this header buffer size is only 8k. 

Syntax:large_client_header_buffers number size;
Default:
large_client_header_buffers 4 8k;
Context:httpserver
What generally happens is that all the cookies used by your site get combined into one header and that may cause you to go over the default limit which is 8192 bytes.

The solution is to just bump the request limit. You can do this globally or just for your site with the LimitRequestFieldSize directive.

This is a tricky error to catch as it only affects people who have cookies over the allotted capacity. Some of your users might experience issues when their cookie size exceeds 8k or like in my case, some pages that set additional cookie value might push you over the limit.

In the first scenario once the user has cookies that are over the limit they wont be able to use the site any more while other users might access the same pages with no problem while their cookies are under the limit. 

Microsoft Websites hosted on IIS issues

In some instances, when authentication is required by a site using Microsoft Live credentials, Bad Request error was showing up because the site is hosted on Microsoft IIS.

Microsoft Sites and affiliated sites use common infrastructure for their credential store using Microsoft Passport, which lets users authenticate to a Microsoft account, an Active Directory account, a Microsoft Azure Active Directory (AD) account, or non-Microsoft service that supports Fast ID Online (FIDO) authentication.
According to the official the Microsoft definition for Bad Request for IIS Web Server for following reason;  


Kerberos authentication token for the user increases in size. The HTTP request that the user sends to the IIS server contains the Kerberos token in the WWW-Authenticate header, and the header size increases as the number of groups goes up.  If the HTTP header or packet size increases past the limits configured in IIS, IIS may reject the request and send this error as the response.

So if the authentication token is too big, it would cause the Bad Request error. However, this problem peaked about 2 years ago and now has subsided, but mentioned for completeness.

So we are left with the remaining reason; generally no funds or time to do the right testing.


The default Microsoft IIS Web Server Header Limits is 64K, which is quite sufficient, but can break, if integrated systems testing is not part of the project plan



For Microsoft IIS HTTP Server, this limit is set by Header Limits <headerLimits> directive (default 64K). The Header Limits <headerLimits>  directive allows the Web server administrator to reduce or increase the limit on the allowed size of an HTTP request header field. The  element of the  collection contains a collection of elements that specify the maximum size in bytes for HTTP headers. 


Chrome Acceptance Header Size  (not the problem)


Chrome can accept a header size of max 256Kb. 


Actual limit seems to be 256KB for the whole HTTP header. Error message appears: "Error 325 (net::ERR_RESPONSE_HEADERS_TOO_BIG): Unknown error."

Wednesday, May 24, 2017

VLC Player Subtitle Hack Attack

Check Point researchers revealed a new attack vector which threatens millions of users worldwide – attack by subtitles. By crafting malicious subtitle files, which are then downloaded by a victim’s media player, attackers can take complete control over any type of device via vulnerabilities found in many popular streaming platforms, including VLC, Kodi (XBMC), Popcorn-Time and strem.io. This includes Macs, Linux,  Windows and Android set-top TV boxes.

Check Points estimates there are approximately 200 million video players and streamers that currently run the vulnerable software, making this one of the most widespread, easily accessed and zero-resistance vulnerability reported in recent years.

VLC has over 186 million downloads of its version 2.2.4 alone, which was released June 5, 2016. Kodi (XBMC) has reached over 10 million unique users per day, and nearly 40 million unique users each month.





























VLC Hack Description

For VLC players, it uses a ParseJSS Null Skip Subtitle Remote Code Execution hijack.
A remote code execution vulnerability exists in VLC Subtitles mechanism. The vulnerability is due to the way VLC parses subtitle files. Successful exploitation could result in arbitrary code execution on the client machine. In the demo video below we see the subtitles essentially activating a TinyVNC connection with the attackers machine, allowing full access for the desktop. 



Action 



Update VLC media player to latest version Version 2.2.6+  immediately

Download now.





Platforms affected


Check all platforms affected at and update
http://blog.checkpoint.com/2017/05/23/hacked-in-translation/


Proof-of-Concept video of a remote hacker taking over your desktop


Tuesday, May 23, 2017

What you should do after a Google Docs Phishing attack?


If you got hit by the massive phishing attack on Google Docs that hit the internet on May 3, it takes phishing to a new level because it was coming from known contacts from you email list. It affected millions of users, since it seemed looked like a legitimate view shared document request.

But there were some dead give-aways, such as message going to hhhhhhhhhhhh?

For those who may have been tricked by the attack and clicked on the phishing link, the attacker potentially had access to the victims' Google accounts and contacts.

Google recommends that users visit https://myaccount.google.com/secureaccount and remove any apps they don't recognize.

Remove granted permissions to others, under Check your account permissions



































Also when you are there if you scroll down to the bottom you see that google stores all your passwords to every site you save you passwords with. 


This is unusual since, these passwords generally should stay on your computer, but Google has decided them to put them in the cloud, for your convenience or an easy back door for Google? 


This is an invasion really of your privacy. It's unexpected browser behavior.

Your Save passwords for sites are stored in the Google Cloud! 

Monday, May 22, 2017

How to find list all Paths Too Long

Using DOS Command

Open a command line shell (Start->Run->cmd.exe  press return)

dir /s /b | sort /r /+261 > pathstoolong.txt
Open pathstoolong.txt to show all path too long for the current logged in user.

cd / 
dir /s /b | sort /r /+261 > pathstoolong.txt
to get to root drive.

This will list all directories and takes a long time. Lines longer than 260 will get to the top of listing. Note, that you must add 1 to SORT column parameter (/+n). Open file in free Notepad++ if really is very large (>1 Gb).

If number of directories returned is large then save you time and use "Path Tool Long Auto Fixer" to fix this automatically.

Sunday, May 21, 2017

How to install and use Windows Performance Toolkit

Windows Performance Toolkit

The Windows Performance Toolkit consists of two independent tools: Windows Performance Recorder (WPR) and Windows Performance Analyzer (WPA). WPA can open any event trace log (ETL) file for analysis. In addition, support is maintained for the previous command-line tool, Xperf. However, Xperfview is no longer supported. All recordings must be opened and analyzed by using WPA.WPA can open any event trace log (ETL) file for analysis.
The following are the system requirements for running Windows Performance Toolkit:
  • Windows Performance Recorder (WPR): Windows 7 or later.
  • Windows Performance Analyzer (WPA): Windows 7 or later with the Microsoft .NET Framework 4.5 or later.

Download the web installer of Windows 10 Assessment and Deployment Kit (Windows ADK) and choose Windows Performance Tookit.





























This is starting point on MSDN of Windows Performance Toolkit
https://msdn.microsoft.com/en-us/windows/hardware/commercialize/test/wpt/index

Good starter for Windows Performance Analyzer
https://msdn.microsoft.com/en-us/windows/hardware/commercialize/test/wpt/windows-performance-analyzer?f=255&MSPPError=-2147217396

https://blogs.technet.microsoft.com/jeff_stokes/2012/09/17/how-to-collect-a-good-boot-trace-on-windows-7/

https://blogs.technet.microsoft.com/jeff_stokes/2013/03/15/xperf-for-the-layman-performance-analysis-unchained-windows-assessment-toolkit-revealed/


Friday, May 19, 2017

Multiple ways to get Grep and Curl for Windows




Download and install official curl for Windows



https://curl.haxx.se/dlwiz/?type=bin&os=Win64&flav=-&ver=*&cpu=x86_64




Choose latest version, and you want SSH enabled as well

curl version: 7.54.0 - SSL enabled SSH enabled
(was latest version as of publishing of this article).

For windows
  1. curl.exe version: 7.54.0 - SSL enabled - here - unzip 
  2. curl.exe version: 7.54.0 - SSL enabled SSH enabled  - here unzip
    Provided by: Viktor Szakts

Copy into C:\Windows\System32

Run CMD.exe or open a command line windows

curl -I -s https://openedx.microsoft.com/


1
2
3
4
5
6
7
C:\Windows\system32>curl -I -s https://openedx.microsoft.com/
HTTP/1.1 500 Internal Server Error
Server: nginx
Date: Mon, 08 May 2017 22:07:54 GMT
Content-Type: text/html
Content-Length: 134
Connection: keep-alive

Line 3 Server : nginx is server manufacturer. 

Or for just the Server name


curl -I -s https://openedx.microsoft.com/  | grep -oP Server:.*

Full command line help documentation here for curl.


Get grep for Windows 


You can get GNUwin32 Grep for Windows, grep.exe version is 2.5.4 here , a little old give that grep 3.0 is officially out. But it comes with an installer and easy to use.

For the latest grep 3.0 you'll have to get Cygwin, but be careful on how you install it.

The default bare bones Cygwin install is 101 MB, but for other packages the full installation can reach 114 Gb.


Or you can get

Utilities and SDK for Subsystem for UNIX-based Applications in Microsoft Windows 7 and Windows Server 2008 R2

https://www.microsoft.com/en-us/download/details.aspx?id=2391
Complete Subsystem for UNIX-based Applications Overview here.

Download here for Windows x32 or x64 or Windows Features in Windows 7.

Choose

Utilities and SDK for Subsystem for UNIX-based Applications_X86.exe and unzip.

in the following directory you can find grep.exe and run it from there

.\Utilities and SDK for Subsystem for UNIX-based Applications_X86\BaseUtils\common

Or install the entire thing.


Windows Server 2003 Resource Kit Tools

Download here, and is a much smaller compact install than Subsystem for UNIX (above).


Tip: Go with this.
Note: grep.exe is replaced by Microsoft® Quick Grep qgrep.exe

Online help for qqrep.exe

List and online help for all Resource Toolkit here.

.
Resource Toolkit also contains head.exe, tail.exe and touch.exe, Duplicate File Finder (dupfinder.exe), File Locator (where.exe) like unix find analog. Brief description of top Resource Toolkit utilities here.







Wednesday, May 17, 2017

Solution to "Path too long" or "The file name(s) would be too long for the destination folder" a free program FastCopy is awesome


WindowsForest
Awards 2015
Grand Prize


FastCopy is free open source C project written by Shirouzu Hiroaki 

Touted as "The Fastest Copy/Delete Software on Windows."  and  "It supports UNICODE and over MAX_PATH (260 characters) file path names."

This means you can over come "paths too long" or "Source Path is too long" "Destination Path is too long"  or  "File cannot be copied" errors in most cases and other similar Path Too Long Tools.

However, this is only 1/4 of the solution. Here's why; 

Firstly, you have to find the paths too long?

Secondly, and most importantly 
this does not solve the problem of the filename itself is too long!!! Which when copied gives you same error. 



OR 


"can't find" error for path that is too long 
You can't rename those files, because you'll get an error. If you successfully copy the file, you are perpetuating the same error. 

You still have to the check and rename each filename to shorter ones manually.





That's why I created this special purpose built software Path Tool Long Auto Fixer app.

FastCopy is a great solution if you know which paths are too long and your number of files/directories you want to move/copy is low. Or you make paths that are too long for testing, the reason I used FC.

To correct the path too long error for good use the Path Tool Long Auto Fixer, the 1st tool on the market to automatically fix long paths in bulk.

Fastcopy won the Japanese Windows Forest Awards 2015 Grand Prize! And it's free.
Unicode support it will work in any language, and is being maintained.

Download here.

FastCopy and long paths problem

This is a great solution, if you know which paths to copy and are too long, and they are relatively few in number.

There's only one down side to coping paths that are tool long to a shorter directory, you still can get the path too long error! The filename can be too long and this tool does not solve that problem. 

You will still get these errors "The file name(s) would be too long for the source/destination folder", next time you copy using Windows Explorer or any back-up solution.

To correct both the path and filename that can be too long automatically use my "Path Tool Long Auto Fixer" tool.


Note: If you are back-up your data online to Onedrive, Gdrive and Dropbox they will complain if the path is too long, so you may want simply to make sure that you don't miss these by fix long paths automatically using "Path Tool Long Auto Fixer" tool.

Or you can use to find all directories and files that are too long in seconds using
Path Tool Long Auto Fixer tool below. There's nothing better than a custom app.


How to automatically fix all your Paths Too Long errors in bulk

  • If you find and fix many paths that are too long, or they are in many different sub-directories and you want to keep your directory structure intact as much as possible, with out moving all those files into a new location. Use Path Tool Long Auto Fixer tool.
  • If you want to verify and find all filenames are that are too long, use Path Tool Long Auto Fixer tool.
  • If you want to never get the "The file name(s) would be too long for the source/destination folder" error, use Path Tool Long Auto Fixer tool.
  • If you want to guarantee not miss a file on your back-up to USB Drive, Onedrive, Gdrive and Dropbox they will complain if the path is too long, so you may want simply to make sure that you don't miss these by fix long paths automatically, use the Path Tool Long Auto Fixer tool.
  • If you want is to have a worry free way of finding an fixing these path tool long errors for good and automatically use the Path Tool Long Auto Fixer Tool


Path Too Long Auto Fixer Tool
I built "Path Tool Long Auto Fixer" tool to find and correct all your long paths AUTOmatically.

It fixes those long paths using a unique algorithm which makes paths and filenames still readable and memorable like removing spaces.
 

Removing space is an obvious optimization that quickly saves allot of space. There are slew of options. Check out a free demo at Path Tool Long Auto Fixer website.

It supports Unicode and therefore works in any language.


Tuesday, May 16, 2017

Bell Canada 2017 Breach of 1.9M Emails represents 66% of Bells customer base - What got released?

The Globe and Mail obtained the hackers statement
"We are releasing a significant portion of Bell.ca data due to the fact that they have failed to [co-operate] with us", says the post, which was published Monday afternoon, several hours before Bell released its apology.

"This shows how Bell doesn't care for its [customers] safety and they could have avoided this public announcement Bell, if you don't [co-operate] more will leak :)." The post contains a link purporting to contain the customer information. It does not clarify what the anonymous poster was seeking co-operation for, or any further intent.

According to Troy Hunt, an Microsoft MVPer who runs haveibeenpwned.com has gotten a hold of the hacked dataset and says that no customer passwords were leaked in Bell Canada May 15, 2017 data breach. 

However, the following customer fields and associated data was leaked. It includes 
Email addresses and usernames.

Check if your Bell email has been hacked in recent Bell 2017 breach at 

Action: Change your username and password for your Bell Billing Account and Bell Email.

This breach represents conservatively 66% of Bells customer base, if we accept assume that Bell Fibe TV subscribers usually bundle their TV and internet, as per http://business.financialpost.com/fp-tech-desk/bell-is-now-canadas-largest-tv-provider-with-more-than-2-7m-subscribers-bce-inc-says. But we do not conclusively if this was from Bell Fibe TV leak or from BCE’s High-Speed Internet (HSI) who's customer base reached 3.4M in 2016. In this case, the breach is about 55% of HSI customer base


Breach: Bell (2017 breach)

Date of breach: 15 May 2017

Number of accounts: 2,231,256

Compromised data


This included customers and Bell employees;


Email addresses, Geographic locations, IP addresses, Job titles, Names, Passwords, Phone numbers, Spoken languages, Survey results, Usernames




Description


In May 2017, the Bell telecommunications company in Canada suffered a data breach resulting in the exposure of millions of customer records.
The data was consequently leaked online with a message from the attacker stating that they were "releasing a significant portion of Bell.ca's data due to the fact that they have failed to cooperate with us" and included a threat to leak more.
The impacted data included over 2 million unique email addresses and 153k survey results dating back to 2011 and 2012. There were also 162 Bell employee records with more comprehensive personal data including names, phone numbers and plain text "passcodes". Bell suffered another breach in 2014 which exposed 40k records.