【故事背景】:
公司某个站点,特别依赖Cookie的使用,而且用的比较狠。在设计之初想当然地以为到达Cookie上限是猴年马月的事儿,没想到时过境迁,这个上限真的来了。
着手改吧,也不想投入太多。于是下面的思路就涌上心头:
【问题】
目前遇到的瓶颈主要是单个Cookie的尺寸超大,在IE下,没有问题,在Firefox下和Chrome下均出现单个Cookie过大,以至于被丢弃的现象。
【思路】
因此为了不侵入旧代码,打算在站点前加一个HttpModule来切分和拼装Cookie。为了保证一次成型,减少投入,仔细阅读了RFC2109文档/RFC6265(http://www.w3.org/Protocols/rfc2109/rfc2109 或http://www.rfc-editor.org/rfc/rfc6265.txt),下面是文档里关于对浏览器Cookie实现的一些说明:
6.3 Implementation Limits Practical user agent implementations have limits on the number and
size of cookies that they can store. In general, user agents' cookie
support should have no fixed limits. They should strive to store as
many frequently-used cookies as possible. Furthermore, general-use
user agents should provide each of the following minimum capabilities
individually, although not necessarily simultaneously: * at least 300 cookies * at least 4096 bytes per cookie (as measured by the size of the
characters that comprise the cookie non-terminal in the syntax
description of the Set-Cookie header) * at least 20 cookies per unique host or domain nameUser agents created for specific purposes or for limited-capacity
devices should provide at least 20 cookies of 4096 bytes, to ensure
that the user can interact with a session-based origin server. The information in a Set-Cookie response header must be retained in
its entirety. If for some reason there is inadequate space to store
the cookie, it must be discarded, not truncated. Applications should use as few and as small cookies as possible, and
they should cope gracefully with the loss of a cookie.
文档中,要求:
至少有300个Cookie总量
每个Cookie至少4096kb
每个域/主机下至少能有20个Cookie
并且还要求:
如果没有什么特殊原因,Cookie就别做啥限制啦!
【实现】
- 难点1:单个Cookie的尺寸限制是包括“Set-Cookie:”以后的部分,而不是Cookie.Value的部分。作为全局Cookie过滤,包括Path/Domain等值事先都是不知道的。我的做法是将.net类库中的生成Cookie的方法翻出来抄一遍。详见“GetSetCookieHeaderString”方法。然后就可以在输出之前先检查一下哪些值出现超长的现象。
- 难点2:不同的浏览器对Cookie的实现不尽相同。经过简单的测试,
- 在Win8.1下:
- IE11:暂时没发现上限
- FireFox(30.0):4145kb
- Chrome(版本 36.0.1985.125 m):4096kb
- 这个世界发展地太快,其实这个问题的最初版本是因为在iOS下发现了兼容性问题,所以不可忽视的是iOS下的容量限制。
- iPad-Safari(iOSversion:7.1.2(11D257)):4125kb?
- iPad-Chrome(iOSversion:7.1.2(11D257)):4125kb?
- iPad-QQ浏览器(iOSversion:7.1.2(11D257)):4125kb?
- 在Win8.1下:
为什么带问号呢?因为这个地方其实最让人头疼。在测试的过程中,我发现Windows下,对Cookie标准理解最正确的是IE,其他浏览器也中规中矩地在做事,顶多是让你做事的时候不得不畏手畏脚。但是iPad则似乎比较调皮,让人捉摸不定。
iPad下的Cookie问题:
- 不论哪个浏览器,似乎都表现出了相同的行为,它们的Cookie是共享的?但其实不是,他们似乎只是遵循了这样的规则,因为清除Cookie和数据后,只有Safari会受到影响。
- 单个Cookie长度是4125kb,但有时候似乎是4123kb,不知道是不是我脑子糊涂后错误地计算了边界值。但是另一个问题,让我放弃了继续追踪这个问题。当一个Cookie超过这个长度后,后续的Cookie就凌乱了。假设你有3个Cookie,如果有一个超长了,后续的Cookie就会丢失。不忙着下结论,继续试验,因此得出了下面的规律。
- 总的Cookie长度,不再像标准所述,每个域至少30个,且每个至少4096kb,总的容量相加的长度似乎也是有限的。我循环了30次,每个Cookie使用了相同的名称,然后控制Cookie的长度,测量出的结果有9510kb这样的长度(所有Cookie相加),但是这不是最终的值,因为一旦后续发生丢失后,不会丢失部分Cookie,要么单个Cookie全部丢失,要么全部保留,因此这个值很难被推算出来。但这个结论让我开始纠结。因为即便所有Cookie都没有超过限制,也没有至少30个Cookie让我存放。
现在讲一下我Windows版本的基本实现思路:
- 在HttpModule.EndRequest事件中,将所有Response的Cookie进行逐一检查,发现太长的,就将其拆成多个,并且在名称上用原始名称+标识符+递增数字的方式进行显示。然后将其添加到Response的Cookie中。为了保持类似IE这样的浏览器仍然能够使用较长的Cookie,而不使用拼拆的(IE仍然是我们的主流客户,任何一次代码调整,在遇到大规模发布的时候,尽量不要改变原来的实现方式,以防止问题大面积爆发),我仍然将原来较长的Cookie放进输出流中。而Chrome这种存储较短Cookie的浏览器,在遇到较长Cookie的时候,会自动丢弃,因此实现了较好的浏览器兼容。
- 在HttpModule.BeginRequest事件中,将所有从浏览器返回的Cookie进行检查,然后将可能发生拼接的进行拼接后供后续程序使用。那么在IE下,如果存在原来名称的Cookie就直接用了,忽略通过拼接回来的Cookie。在Chrome下,发现原本传下去较长的Cookie已经丢失,因此只能使用拼接后的结果,因此也避免的Cookie丢失的问题。
但这一切在iPad下,就令人头疼了。因为只有发现超长的时候,才进行拆分,而且同时为了向上兼容,那么两个4096就已经接近其上限了,那么在这样的场景下,再去使用,就会发生Cookie莫名其妙随机丢失的情况,而实验的结果也是如此。因此,可能就需要(还没时间去测试)首先判断浏览器是不是来自于移动设备?iOS?iPad?等,然后再进行Cookie实现,那么这个方案就会更复杂,但这个上限给人的感觉是,跑得了初一跑不了十五。很快这个Cookie值就会穿过枷锁,让你头疼欲裂。
真是一失足成千古恨,任何一个依赖客户端的场景都会因为时间地推移而变得无奈。关于iPad Cookie相关的问题,如果有园友知情的也请不吝留言告知。
附上代码:
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Web; namespace LargeCookieModule
{
public class CookieHelper
{
/// <summary>
/// firefox.total:4145
/// chrome.total:4096
/// </summary>
private const int CONST_MAX_COOKIE_BYTES = ;
private const string CONST_BEYOND_SEPARATOR = "_C_SPO_";
/// <summary>
/// http://www.rfc-editor.org/rfc/rfc6265.txt
/// At least 4096 bytes per cookie (as measured by the sum of the length of the cookie's name, value, and attributes).
/// At least 50 cookies per domain.
/// At least 3000 cookies total.
/// 目前考虑最多2位数来解决cookie溢出的问题。
/// 另CONST_BEYOND_SEPARATOR.Length位用来解决SEPARATOR的问题。
/// </summary>
private const int CONST_BEYOND_CHARS = ( + );
private static System.Text.Encoding defaultEncoding
{
get
{
return System.Text.Encoding.UTF8;
}
} public static string GetSetCookieHeaderString(HttpContext context, HttpCookie cookie)
{
string name = cookie.Name;
string value = cookie.Value;
string domain = cookie.Domain;
DateTime expires = cookie.Expires;
string path = cookie.Path;
bool secure = cookie.Secure;
bool httpOnly = cookie.HttpOnly; StringBuilder stringBuilder = new StringBuilder();
if (!string.IsNullOrEmpty(name))
{
stringBuilder.Append(name);
stringBuilder.Append('=');
}
if (!string.IsNullOrEmpty(value))
{
stringBuilder.Append(value);
}
if (!string.IsNullOrEmpty(domain))
{
stringBuilder.Append("; domain=");
stringBuilder.Append(domain);
}
if (expires != DateTime.MinValue)
{
stringBuilder.Append("; expires=");
stringBuilder.Append(FormatHttpCookieDateTime(expires));
}
if (!string.IsNullOrEmpty(path))
{
stringBuilder.Append("; path=");
stringBuilder.Append(path);
}
if (secure)
{
stringBuilder.Append("; secure");
}
if (httpOnly && SupportsHttpOnly(context))
{
stringBuilder.Append("; HttpOnly");
}
return stringBuilder.ToString();
} private static bool SupportsHttpOnly(HttpContext context)
{
if (context != null && context.Request != null)
{
HttpBrowserCapabilities browser = context.Request.Browser;
return browser != null && (browser.Type != "IE5" || browser.Platform != "MacPPC");
}
return false;
} /// <summary>
/// equals to HttpUtility.FormatHttpCookieDateTime
/// for java system to rewrite code.
/// </summary>
/// <param name="dt"></param>
/// <returns></returns>
private static string FormatHttpCookieDateTime(DateTime dt)
{
if (dt < DateTime.MaxValue.AddDays(-1.0) && dt > DateTime.MinValue.AddDays(1.0))
{
dt = dt.ToUniversalTime();
}
return dt.ToString("ddd, dd-MMM-yyyy HH':'mm':'ss 'GMT'", DateTimeFormatInfo.InvariantInfo);
} /// <summary>
/// 是否是合法的Cookies(目前只考虑长度)
/// 目前暂时只知道用System.Text.Encoding.UTF8来解决。无法获取客户端存储的情况。
/// </summary>
/// <param name="context"></param>
/// <param name="cookie"></param>
/// <returns></returns>
public static bool IsLengthLegalCookie(HttpContext context, HttpCookie cookie)
{
if (cookie == null)
return false;
string cookieString = CookieHelper.GetSetCookieHeaderString(context, cookie); if (cookieString != null && defaultEncoding.GetByteCount(cookieString) <= CONST_MAX_COOKIE_BYTES)
{
return true;
} return false;
} /// <summary>
/// 获取除了Value(name+value)值以外的字符长度
/// </summary>
/// <param name="context"></param>
/// <param name="cookie"></param>
/// <returns></returns>
private static int GetLengthExceptValue(HttpContext context, HttpCookie cookie)
{
if (cookie == null)
return ;
string cookieString = CookieHelper.GetSetCookieHeaderString(context, cookie);
if (!string.IsNullOrEmpty(cookieString))
{
int index = cookieString.IndexOf(';');
// if index equals to -1 result is - (-1) - 1.
return cookieString.Length - index - ;
}
return ;
} public static List<HttpCookie> GetCookies(HttpCookieCollection cookies)
{
Dictionary<string, HttpCookie> resultCookies = new Dictionary<string, HttpCookie>();
SeparatorSortedDictionary<HttpCookie> separatorCookies = new SeparatorSortedDictionary<HttpCookie>(); if (cookies != null && cookies.AllKeys != null && cookies.AllKeys.Length > )
{
foreach (string key in cookies.AllKeys)
{
HttpCookie cookie = cookies[key];
if (cookie != null && !string.IsNullOrEmpty(cookie.Name))
{
int separatorIndex = cookie.Name.IndexOf(CONST_BEYOND_SEPARATOR);
if (separatorIndex != -)
{
string prefixName = cookie.Name.Substring(, separatorIndex);
string sOrder = cookie.Name.Substring(prefixName.Length + CONST_BEYOND_SEPARATOR.Length);
int iOrder = ;
if (int.TryParse(sOrder, out iOrder))
{
separatorCookies.Add(prefixName, iOrder, cookie);
}
}
else
{
resultCookies[cookie.Name] = cookie;
}
}
}
List<string> separatorKeys = separatorCookies.Keys;
foreach (string key in separatorKeys)
{
if(resultCookies.ContainsKey(key))
continue;
HttpCookie joinCookie = separatorCookies.Join(key,
(c) => c.Value,
(s, c) => { c.Value = s; return c; },
CloneHttpCookie);
resultCookies[key] = joinCookie;
}
}
return resultCookies.Values.ToList();
} public static List<HttpCookie> GetSetCookies(HttpContext context, HttpCookie cookie)
{
if (IsLengthLegalCookie(context, cookie))
{
List<HttpCookie> cookies = new List<HttpCookie>();
cookies.Add(cookie);
return cookies;
}
else
{
string name = cookie.Name;
string value = cookie.Value;
int maxValueCount = CONST_MAX_COOKIE_BYTES - CONST_BEYOND_CHARS - GetLengthExceptValue(context, cookie) - name.Length - /*name=value后面的分号*/;
if (maxValueCount > )
{
byte[] bValue = defaultEncoding.GetBytes(value);
int iEachChunk = , iChunkNum, iChunkCount = ;
SortedList<int, byte[]> chunks = new SortedList<int, byte[]>();
iChunkCount = bValue.Length / maxValueCount + ;
int iChunkLastIndex = iChunkCount-;
for (iChunkNum = ; iChunkNum < iChunkCount; ++iChunkNum)
{
byte[] chunk = null;
if (iChunkNum != iChunkLastIndex)
{
chunk = new byte[maxValueCount];
}
else
{
chunk = new byte[bValue.Length % maxValueCount];
}
for (iEachChunk = ; iEachChunk < chunk.Length; ++iEachChunk)
{
chunk[iEachChunk] = bValue[iChunkNum * maxValueCount + iEachChunk];
}
chunks.Add(iChunkNum, chunk);
}
List<HttpCookie> cookies = new List<HttpCookie>();
foreach (KeyValuePair<int, byte[]> item in chunks)
{
string itemName = name + CONST_BEYOND_SEPARATOR + item.Key.ToString();
string itemValue = defaultEncoding.GetString(item.Value);
HttpCookie cloneCookie = CloneHttpCookie(cookie);
cloneCookie.Name = itemName;
cloneCookie.Value = itemValue;
cookies.Add(cloneCookie);
}
return cookies;
}
}
return null;
} public static bool IsSeparatorCookie(HttpCookie cookie)
{
if (cookie == null)
return false;
if (!string.IsNullOrEmpty(cookie.Name) && cookie.Name.Contains(CONST_BEYOND_SEPARATOR))
{
return true;
}
return false;
} private static HttpCookie CloneHttpCookie(HttpCookie cookie)
{
HttpCookie cloneCookie = new HttpCookie(cookie.Name);
cloneCookie.Value = cookie.Value;
cloneCookie.Path = cookie.Path;
cloneCookie.Secure = cookie.Secure;
cloneCookie.HttpOnly = cookie.HttpOnly;
cloneCookie.Domain = cookie.Domain;
cloneCookie.Expires = cookie.Expires;
return cloneCookie;
} public static void FilterRequestCookies(HttpContext context)
{
if (context != null)
{
IList<HttpCookie> cookies = CookieHelper.GetCookies(context.Request.Cookies);
if (cookies != null)
{
foreach (HttpCookie subCookie in cookies)
{
if (context.Request.Cookies[subCookie.Name] == null)
{
context.Request.Cookies.Add(subCookie);
}
}
}
}
} public static void FilterResponseCookies(HttpContext context)
{
if (context != null && context.Response.Cookies != null && context.Response.Cookies.Keys.Count > )
{
int keyCount = context.Response.Cookies.Keys.Count;
if (keyCount > )
{
string[] keys = new string[keyCount];
for (int i = ; i < keyCount; ++i)
{
keys[i] = context.Response.Cookies.Keys[i];
} foreach (string key in keys)
{
HttpCookie everyCookie = context.Response.Cookies[key]; IList<HttpCookie> cookies = CookieHelper.GetSetCookies(context, everyCookie);
if (cookies != null && cookies.Count > /*means it is separator cookie or the raw cookie is not legal in length.*/)
{
foreach (HttpCookie subCookie in cookies)
{
if (IsSeparatorCookie(subCookie))
{
context.Response.AppendCookie(subCookie);
}
}
}
}
}
}
} internal class SeparatorSortedDictionary<T> : Dictionary<string, SortedList<int, T>>
{
Dictionary<string, SortedList<int, T>> _innerStore = new Dictionary<string, SortedList<int, T>>();
public void Add(string name, int index, T value)
{
if (_innerStore != null)
{
if (!_innerStore.ContainsKey(name))
_innerStore[name] = new SortedList<int, T>();
SortedList<int, T> itemCollection = _innerStore[name];
if (!itemCollection.ContainsKey(index))
{
itemCollection.Add(index, value);
}
}
} public T Join(string name, Func<T, string> toStringFunc, Func<string, T, T> fromStringFunc, Func<T, T> cloneFunc)
{
if (_innerStore != null)
{
if (_innerStore.ContainsKey(name))
{
SortedList<int, T> itemCollection = _innerStore[name];
string value = string.Empty;
T firstElement = default(T);
bool getFirstElement = false;
foreach (T itemValue in itemCollection.Values)
{
if (!getFirstElement)
{
firstElement = itemValue;
getFirstElement = true;
}
value += toStringFunc(itemValue);
}
T cloneItem = cloneFunc(firstElement);
cloneItem = fromStringFunc(value, cloneItem);
return cloneItem;
}
}
return default(T);
} public List<string> Keys
{
get
{
List<string> result = new List<string>();
if (_innerStore != null)
{
foreach (string key in _innerStore.Keys)
{
result.Add(key);
}
}
return result;
}
}
}
}
}
调用代码:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Web; namespace LargeCookieModule
{
public class LargeCookieModule : IHttpModule
{
public void Dispose()
{
throw new NotImplementedException();
} public void Init(HttpApplication context)
{
context.BeginRequest += context_BeginRequest;
context.EndRequest += context_EndRequest;
} void context_BeginRequest(object sender, EventArgs e)
{
HttpApplication app = sender as HttpApplication;
if (app != null)
{
if (app.Context.Request.ContentEncoding != null)
app.Context.Response.Write("app.Context.Request.ContentEncoding:" + app.Context.Request.ContentEncoding.ToString() + "<br />"); app.Context.Response.Write("<br />-----------(start)raw request cookies, context_BeginRequest:--------------<br />"); foreach (string key in app.Request.Cookies.AllKeys)
{
HttpCookie cookie = app.Request.Cookies[key];
if (cookie != null)
{
string cookieString = CookieHelper.GetSetCookieHeaderString(app.Context, cookie);
app.Context.Response.Write("context_BeginRequest:" + cookieString + "<br />");
app.Context.Response.Write("context_BeginRequest.cookieLength:" + cookieString.Length + "<br />");
app.Context.Response.Write("context_BeginRequest.GetByteCount:" + System.Text.Encoding.UTF8.GetByteCount(cookieString) + "<br />");
}
} app.Context.Response.Write("<br />-----------(end)raw request cookies, context_BeginRequest:--------------<br /><br />"); app.Context.Response.Write("<br />-----------(start)merged by CookieHelper.cs, context_BeginRequest:--------------<br />");
// 唯一一句用来过滤
CookieHelper.FilterRequestCookies(app.Context);
foreach (string key in app.Request.Cookies.AllKeys)
{
HttpCookie cookie = app.Request.Cookies[key];
if (cookie != null)
{
string cookieString = CookieHelper.GetSetCookieHeaderString(app.Context, cookie);
app.Context.Response.Write("context_BeginRequest:" + cookieString + "<br />");
app.Context.Response.Write("context_BeginRequest.cookieLength:" + cookieString.Length + "<br />");
app.Context.Response.Write("context_BeginRequest.GetByteCount:" + System.Text.Encoding.UTF8.GetByteCount(cookieString) + "<br />");
}
}
app.Context.Response.Write("<br />-----------(end)merged by CookieHelper.cs, context_BeginRequest:--------------<br /><br />");
}
} void context_EndRequest(object sender, EventArgs e)
{
//HttpApplication app = sender as HttpApplication;
//if (app != null)
//{
// // CookieHelper.FilterResponseCookies(app.Context);
// app.Context.Response.Write("<br />-----------(start)response the merged cookies to the client, context_EndRequest:--------------<br />");
// foreach (string key in app.Response.Cookies.AllKeys)
// {
// HttpCookie cookie = app.Response.Cookies[key];
// if (cookie != null)
// {
// IList<HttpCookie> cookies = CookieHelper.GetSetCookies(app.Context, cookie);
// if (cookies != null)
// {
// foreach (HttpCookie subCookie in cookies)
// {
// string cookieString = CookieHelper.GetSetCookieHeaderString(app.Context, subCookie);
// app.Context.Response.AppendCookie(subCookie);
// app.Context.Response.Write("context_EndRequest:" + cookieString + "<br />");
// app.Context.Response.Write("context_EndRequest.cookieLength:" + cookieString.Length + "<br />");
// app.Context.Response.Write("context_EndRequest.GetByteCount:" + System.Text.Encoding.UTF8.GetByteCount(cookieString) + "<br />");
// }
// }
// }
// }
// app.Context.Response.Write("<br />-----------(end)response the merged cookies to the client, context_EndRequest:--------------<br /><br />");
//} HttpApplication app = sender as HttpApplication;
if (app != null)
{
app.Context.Response.Write("<br />-----------(start)response the all cookies to the client, context_EndRequest:--------------<br />"); CookieHelper.FilterResponseCookies(app.Context); List<HttpCookie> reverseList = new List<HttpCookie>();
List<string> keys = app.Response.Cookies.AllKeys.ToList();
foreach (string key in keys)
{
HttpCookie cookie = app.Response.Cookies[key];
if (cookie != null)
{
reverseList.Add(cookie);
string cookieString = CookieHelper.GetSetCookieHeaderString(app.Context, cookie);
app.Context.Response.Write("context_EndRequest:" + cookieString + "<br />");
app.Context.Response.Write("context_EndRequest.cookieLength:" + cookieString.Length + "<br />");
app.Context.Response.Write("context_EndRequest.GetByteCount:" + System.Text.Encoding.UTF8.GetByteCount(cookieString) + "<br />");
}
} app.Context.Response.Write("<br />-----------(end)response the all cookies to the client, context_EndRequest:--------------<br />");
}
}
}
}
其中BeginRequest和EndRequest中其实只需要调用:
CookieHelper.FilterRequestCookies(app.Context);
CookieHelper.FilterResponseCookies(app.Context);
以上其余代码都是用来测试的。
另外附上iPad测试的代码:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls; namespace TestIPadCookie
{
public partial class Default : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
for (int j = ; j < ; ++j)
{
string text = string.Empty;
string cookieString = string.Empty;
HttpCookie cookie = new HttpCookie("ipadTest" + j.ToString().PadLeft(, ''));
cookie.Expires = DateTime.Now.AddDays();
for (int i = ; i < ; ++i)
{
text += 'c';
cookie.Value = text;
cookieString = LargeCookieModule.CookieHelper.GetSetCookieHeaderString(HttpContext.Current, cookie);
if (System.Text.Encoding.UTF8.GetByteCount(cookieString) >= )
{
break;
}
}
HttpContext.Current.Response.AppendCookie(cookie);
HttpContext.Current.Response.Write("cookieLength=" + cookieString.Length.ToString() + "<br />");
HttpContext.Current.Response.Write("cookieValueLength=" + text.Length.ToString()
+ ", gap = " + Math.Abs(cookieString.Length - text.Length) + "<br />");
HttpContext.Current.Response.Write("cookieString=" + cookieString + "<br />");
HttpContext.Current.Response.Write("<br />");
}
}
}
}
源代码打包下载:(点击这里)
http://files.cnblogs.com/volnet/WebAppLargeCookieModule.zip