bad request 'limit request headers fields size' opendx microsoft

Some Microsoft sites returns the following HTTP error in Google Chrome only 

Bad Request  Error parsing headers: 'limit request headers fields size'

Microsoft's OpenDx  learning site returns a Bad Request (May 4th-6th, 2017, May 25, 2017) in Chrome.

https://openedx.microsoft.com/


This is new Microsoft Learning site hosted on Azure (Microsoft's cloud service) using Open edX an open source training platform from Harvard and MIT. 

Explanation

This is caused by a browser initial accessing the page on the server. The server is asking for information in an HTTP request from the browser, usually this is an cookie that in part contains your authorization information. Each time you access the site the cookie is updated and depending on its design, it can grow. If it grows two large then you can get a Bad Request error from the server. If you get this error, then the new cookie information is never set for this domain.


This can also be caused by changes to code in the page setting a new cookie definition, or a change to web server configuration. If web server is being rebuilt nightly, this this could cause this affect the cookie definition, invalidating previous cookies.

Solution

Clear your Chrome cookies with a free tool (see my post on this) for opendx.microsoft.com to verify and delete them. You do not have to clear all your cookies for all sites. 

It seems for the openedx.microsoft.com Chrome is not expiring its cookies properly. 


But the error may persist, depend where you are on the build cycle of this site. 

I will demonstrate that this is not a cookie issue for me. On May 4th, 2017 and May 25, 2017 https://openedx.microsoft.com/ had this error in Chrome. 


I cleared my cookies for opendx.microsoft.com and still I could not get into the site. 

I revisited the page and check the cookies for opendx.microsoft.com and Chrome doesn't even have a chance to set a cookie. 

However in Mozilla Firefox, for the site https://openedx.microsoft.com/ sets a csrftoken cookie, but not in Chrome. 

In this instance, it probably is a server configuration error. Following that lead, I found openedx.microsoft.com is running on Nginx server.  Read my post on how to get the server manufacturer using curl for windows, to check this issue for other servers.

Nginx servers have a small default header size, see below. 


In the end, this error will be resolved quickly, usually in the same day, because the site cannot negate Chrome representing 60% of market share for long.

But in the meantime, use Microsoft IE or Mozilla Firefox browser instead, until this is server configuration is fixed.

NGIX Server Hosted Websites cookie issue 

Lately the Bad Request error is occurring many times on Microsoft and Microsoft Affiliated Websites when using Chrome.

One began to suspect that this was a ploy by Microsoft to generate traffic for IE. However, most likely its just poor developer testing.

For sites hosted on NGIX servers, this header buffer size is only 8k. 

Syntax:	large_client_header_buffers number size;
Default:	large_client_header_buffers 4 8k;
Context:	http, server

What generally happens is that all the cookies used by your site get combined into one header and that may cause you to go over the default limit which is 8192 bytes.

The solution is to just bump the request limit. You can do this globally or just for your site with the LimitRequestFieldSize directive.

This is a tricky error to catch as it only affects people who have cookies over the allotted capacity. Some of your users might experience issues when their cookie size exceeds 8k or like in my case, some pages that set additional cookie value might push you over the limit.

In the first scenario once the user has cookies that are over the limit they wont be able to use the site any more while other users might access the same pages with no problem while their cookies are under the limit. 

Microsoft Websites hosted on IIS issues

In some instances, when authentication is required by a site using Microsoft Live credentials, Bad Request error was showing up because the site is hosted on Microsoft IIS.

Microsoft Sites and affiliated sites use common infrastructure for their credential store using Microsoft Passport, which lets users authenticate to a Microsoft account, an Active Directory account, a Microsoft Azure Active Directory (AD) account, or non-Microsoft service that supports Fast ID Online (FIDO) authentication.According to the official the Microsoft definition for Bad Request for IIS Web Server for following reason;  

Kerberos authentication token for the user increases in size. The HTTP request that the user sends to the IIS server contains the Kerberos token in the WWW-Authenticate header, and the header size increases as the number of groups goes up.  If the HTTP header or packet size increases past the limits configured in IIS, IIS may reject the request and send this error as the response.

So if the authentication token is too big, it would cause the Bad Request error. However, this problem peaked about 2 years ago and now has subsided, but mentioned for completeness.

So we are left with the remaining reason; generally no funds or time to do the right testing.

The default Microsoft IIS Web Server Header Limits is 64K, which is quite sufficient, but can break, if integrated systems testing is not part of the project plan

For Microsoft IIS HTTP Server, this limit is set by Header Limits <headerLimits> directive (default 64K). The Header Limits <headerLimits>  directive allows the Web server administrator to reduce or increase the limit on the allowed size of an HTTP request header field. The  element of the  collection contains a collection of elements that specify the maximum size in bytes for HTTP headers. 

Chrome Acceptance Header Size  (not the problem)

Chrome can accept a header size of max 256Kb. 

Actual limit seems to be 256KB for the whole HTTP header. Error message appears: "Error 325 (net::ERR_RESPONSE_HEADERS_TOO_BIG): Unknown error."

VLC Player Subtitle Hack Attack

Check Point researchers revealed a new attack vector which threatens millions of users worldwide – attack by subtitles. By crafting malicious subtitle files, which are then downloaded by a victim’s media player, attackers can take complete control over any type of device via vulnerabilities found in many popular streaming platforms, including VLC, Kodi (XBMC), Popcorn-Time and strem.io. This includes Macs, Linux,  Windows and Android set-top TV boxes.

Check Points estimates there are approximately 200 million video players and streamers that currently run the vulnerable software, making this one of the most widespread, easily accessed and zero-resistance vulnerability reported in recent years.

VLC has over 186 million downloads of its version 2.2.4 alone, which was released June 5, 2016. Kodi (XBMC) has reached over 10 million unique users per day, and nearly 40 million unique users each month.

VLC Hack Description

For VLC players, it uses a ParseJSS Null Skip Subtitle Remote Code Execution hijack.

A remote code execution vulnerability exists in VLC Subtitles mechanism. The vulnerability is due to the way VLC parses subtitle files. Successful exploitation could result in arbitrary code execution on the client machine. In the demo video below we see the subtitles essentially activating a TinyVNC connection with the attackers machine, allowing full access for the desktop. 

Action 

Update VLC media player to latest version Version 2.2.6+  immediately

Download now.

Platforms affected

Check all platforms affected at and update
http://blog.checkpoint.com/2017/05/23/hacked-in-translation/


Proof-of-Concept video of a remote hacker taking over your desktop

What you should do after a Google Docs Phishing attack?

If you got hit by the massive phishing attack on Google Docs that hit the internet on May 3, it takes phishing to a new level because it was coming from known contacts from you email list. It affected millions of users, since it seemed looked like a legitimate view shared document request.

But there were some dead give-aways, such as message going to hhhhhhhhhhhh?

For those who may have been tricked by the attack and clicked on the phishing link, the attacker potentially had access to the victims' Google accounts and contacts.

Google recommends that users visit https://myaccount.google.com/secureaccount and remove any apps they don't recognize.

Remove granted permissions to others, under Check your account permissions

Also when you are there if you scroll down to the bottom you see that google stores all your passwords to every site you save you passwords with. 

This is unusual since, these passwords generally should stay on your computer, but Google has decided them to put them in the cloud, for your convenience or an easy back door for Google? 

This is an invasion really of your privacy. It's unexpected browser behavior.
Your Save passwords for sites are stored in the Google Cloud! 

How to find list all Paths Too Long

Using DOS Command

Open a command line shell (Start->Run->cmd.exe  press return)

dir /s /b | sort /r /+261 > pathstoolong.txt

Open pathstoolong.txt to show all path too long for the current logged in user.

cd / 

dir /s /b | sort /r /+261 > pathstoolong.txt

to get to root drive.

This will list all directories and takes a long time. Lines longer than 260 will get to the top of listing. Note, that you must add 1 to SORT column parameter (/+n). Open file in free Notepad++ if really is very large (>1 Gb).

If number of directories returned is large then save you time and use "Path Tool Long Auto Fixer" to fix this automatically.

How to install and use Windows Performance Toolkit

Windows Performance Toolkit

The Windows Performance Toolkit consists of two independent tools: Windows Performance Recorder (WPR) and Windows Performance Analyzer (WPA). WPA can open any event trace log (ETL) file for analysis. In addition, support is maintained for the previous command-line tool, Xperf. However, Xperfview is no longer supported. All recordings must be opened and analyzed by using WPA.WPA can open any event trace log (ETL) file for analysis.

The following are the system requirements for running Windows Performance Toolkit:

• Windows Performance Recorder (WPR): Windows 7 or later.
• Windows Performance Analyzer (WPA): Windows 7 or later with the Microsoft .NET Framework 4.5 or later.

Download the web installer of Windows 10 Assessment and Deployment Kit (Windows ADK) and choose Windows Performance Tookit.

This is starting point on MSDN of Windows Performance Toolkit
https://msdn.microsoft.com/en-us/windows/hardware/commercialize/test/wpt/index

Good starter for Windows Performance Analyzer
https://msdn.microsoft.com/en-us/windows/hardware/commercialize/test/wpt/windows-performance-analyzer?f=255&MSPPError=-2147217396

https://blogs.technet.microsoft.com/jeff_stokes/2012/09/17/how-to-collect-a-good-boot-trace-on-windows-7/

https://blogs.technet.microsoft.com/jeff_stokes/2013/03/15/xperf-for-the-layman-performance-analysis-unchained-windows-assessment-toolkit-revealed/

Multiple ways to get Grep and Curl for Windows

Getting CURL for Windows


Download and install official curl for Windows


https://curl.se/dlwiz/?type=bin



Choose latest version, and you want SSH enabled as well

curl version: 7.54.0 - SSL enabled SSH enabled

(was latest version as of publishing of this article).

For windows

1. curl.exe version: 7.54.0 - SSL enabled - here 
2. curl.exe version: 7.54.0 - SSL enabled SSH enabled  - here 
   Provided by: Viktor Szakts
3. https://curl.haxx.se/dlwiz/?type=bin&os=Win64&flav=-&ver=*&cpu=x86_64
   

Copy into C:\Windows\System32

Run CMD.exe or open a command line windows

curl -I -s https://openedx.microsoft.com/

1 2 3 4 5 6 7

C:\Windows\system32>curl -I -s https://openedx.microsoft.com/ HTTP/1.1 500 Internal Server Error Server: nginx Date: Mon, 08 May 2017 22:07:54 GMT Content-Type: text/html Content-Length: 134 Connection: keep-alive

Line 3 Server : nginx is server manufacturer. 

Or for just the Server name

curl -I -s https://openedx.microsoft.com/  | grep -oP Server:.*

Full command line help documentation here for curl.


Get grep for Windows 

1) Official GNU Port

Grep - GNU Project - Free Software Foundation

Grep for Windows (sourceforge.net) - sourceforge.net was dubious in the past, check 

Cygwin - large install 
2) You can get GNUwin32 Grep for Windows, grep.exe version is 2.5.4 here , a little old give that grep 3.0 is officially out. But it comes with an installer and easy to use.

For the latest grep 3.0 you'll have to get Cygwin, but be careful on how you install it.

The default bare bones Cygwin install is 101 MB, but for other packages the full installation can reach 114 Gb.



3) Utilities and SDK for Subsystem for UNIX-based Applications in Microsoft Windows 7 and Windows Server 2008 R2 - can extra only grep cmd

https://www.microsoft.com/en-us/download/details.aspx?id=2391
Complete Subsystem for UNIX-based Applications Overview here.

Download here for Windows x32 or x64 or Windows Features in Windows 7.

Choose
Utilities and SDK for Subsystem for UNIX-based Applications_X86.exe and unzip.

in the following directory you can find grep.exe and run it from there
.\Utilities and SDK for Subsystem for UNIX-based Applications_X86\BaseUtils\common

Or install the entire thing.


Windows Server 2003 Resource Kit Tools

4) Download here, and is a much smaller compact install than Subsystem for UNIX (above).

Tip: Go with this.
Note: grep.exe is replaced by Microsoft® Quick Grep qgrep.exe

Online help for qqrep.exe. 

List and online help for all Resource Toolkit here.

.
Resource Toolkit also contains head.exe, tail.exe and touch.exe, Duplicate File Finder (dupfinder.exe), File Locator (where.exe) like unix find analog. Brief description of top Resource Toolkit utilities here.

Get GUI grep for Windows 
1) grepWin - is a GUI version but very fast, get it from the source on Github.

grepWin - Stefans Tools (stefankueng.com)

Solution to "Path too long" or "The file name(s) would be too long for the destination folder" a free program FastCopy is awesome

Ever see this error? 

OR 

"can't find" error for path that is too long 

Here's your solution to this; 

How to automatically find, report, fix all your Paths Too Long errors in bulk

• If you find and fix many paths that are too long, or they are in many different sub-directories and you want to keep your directory structure intact as much as possible, with out moving all those files into a new location. Use Path Tool Long Auto Fixer tool.
• If you want to verify and find all filenames are that are too long, use Path Tool Long Auto Fixer tool.
• If you want to never get the "The file name(s) would be too long for the source/destination folder" error, use Path Tool Long Auto Fixer tool.
• If you want to guarantee not miss a file on your back-up to USB Drive, Onedrive, Gdrive and Dropbox they will complain if the path is too long, so you may want simply to make sure that you don't miss these by fix long paths automatically, use the Path Tool Long Auto Fixer tool.
• If you want is to have a worry free way of finding an fixing these path tool long errors for good and automatically use the Path Tool Long Auto Fixer Tool

Path Too Long Auto Fixer Tool
I built "Path Tool Long Auto Fixer" tool to find and correct all your long paths AUTOmatically.

It fixes those long paths using a unique algorithm which makes paths and filenames still readable and memorable like removing spaces. 

Removing space is an obvious optimization that quickly saves allot of space. There are slew of options. Check out a free demo at Path Tool Long Auto Fixer website.

It supports Unicode and therefore works in any language.

Bell Canada 2017 Breach of 1.9M Emails represents 66% of Bells customer base - What got released?

The Globe and Mail obtained the hackers statement

"We are releasing a significant portion of Bell.ca data due to the fact that they have failed to [co-operate] with us", says the post, which was published Monday afternoon, several hours before Bell released its apology.

"This shows how Bell doesn't care for its [customers] safety and they could have avoided this public announcement Bell, if you don't [co-operate] more will leak :)." The post contains a link purporting to contain the customer information. It does not clarify what the anonymous poster was seeking co-operation for, or any further intent.

According to Troy Hunt, an Microsoft MVPer who runs haveibeenpwned.com has gotten a hold of the hacked dataset and says that no customer passwords were leaked in Bell Canada May 15, 2017 data breach. 

However, the following customer fields and associated data was leaked. It includes Email addresses and usernames.

Check if your Bell email has been hacked in recent Bell 2017 breach at 

https://haveibeenpwned.com

Action: Change your username and password for your Bell Billing Account and Bell Email.

This breach represents conservatively 66% of Bells customer base, if we accept assume that Bell Fibe TV subscribers usually bundle their TV and internet, as per http://business.financialpost.com/fp-tech-desk/bell-is-now-canadas-largest-tv-provider-with-more-than-2-7m-subscribers-bce-inc-says. But we do not conclusively if this was from Bell Fibe TV leak or from BCE’s High-Speed Internet (HSI) who's customer base reached 3.4M in 2016. In this case, the breach is about 55% of HSI customer base. 

Breach: Bell (2017 breach)

Date of breach: 15 May 2017

Number of accounts: 2,231,256

Compromised data

This included customers and Bell employees;

Email addresses, Geographic locations, IP addresses, Job titles, Names, Passwords, Phone numbers, Spoken languages, Survey results, Usernames

Description

In May 2017, the Bell telecommunications company in Canada suffered a data breach resulting in the exposure of millions of customer records.

The data was consequently leaked online with a message from the attacker stating that they were "releasing a significant portion of Bell.ca's data due to the fact that they have failed to cooperate with us" and included a threat to leak more.

The impacted data included over 2 million unique email addresses and 153k survey results dating back to 2011 and 2012. There were also 162 Bell employee records with more comprehensive personal data including names, phone numbers and plain text "passcodes". Bell suffered another breach in 2014 which exposed 40k records.

Intel Active Management Technology - AMT Flaw Password Bypass Threat

The critical Active Management Technology - AMT - flaw present in the firmware running on many Intel chips since 2010 is worse than feared, security researchers warn. In particular, the flaw can be easily exploited to allow a remote attacker to take control of vulnerable systems without even having to enter a password.

This mainly affects the Intel vPro branded chipset.


Here's Intel official response

1. Intel Active Management Technology, Intel Small Business Technology, and Intel Standard Manageability Escalation of Privilege - read this
2. INTEL-SA-00075 Detection Software and Guide - how to solve it

Action choose 2 from above;
1. Download MSI install, double click to install. 

2. When completed search for Intel-SA-00075 Discovery Tool

3. Run it.
4. My results is Unknown

As per INTEL-SA-00075 Detection Guide recommends further investigating manufacturers link below.

In my case Gateway is owned by Acer, so I have to check link below, but is not supported :( 

As Intel becomes aware of computer maker schedules for updated firmware this list will be updated:

• HP Inc. - http://www8.hp.com/us/en/intelmanageabilityissue.html
• HP Enterprise - http://h22208.www2.hpe.com/eginfolib/securityalerts/CVE-2017-5689-Intel/CVE-2017-5689.html
• Lenovo - https://support.lenovo.com/us/en/product_security/LEN-14963
• Fujitsu - http://support.ts.fujitsu.com/content/Intel_Firmware.asp
• Dell Client - http://en.community.dell.com/techcenter/extras/m/white_papers/20443914
• Dell EMC - http://en.community.dell.com/techcenter/extras/m/white_papers/20443937
• Acer - https://us.answers.acer.com/app/answers/detail/a_id/47162
• Asus - https://www.asus.com/News/uztEkib4zFMHCn5r
• Panasonic - http://pc-dl.panasonic.co.jp/itn/info/osinfo20170512.html
• Intel – NUC, Compute Stick and Desktop Boards

Intel Processor Identification Utility - Windows* Version produces following result indicates VT technology but no AMT.