2/03/2009

WebDAV MOVE method fails with 502 Bad Gateway Error with HTTPS

I was recently writing a Javascript piece of code to move a bunch files from one location to another in the same WebDAV repository. It worked all well in the lower systems but when I tried the same thing in the Production environment I got the "502 Bad Gateway" exception. I thought probably I was doing some mistake and I tried to use the WebDAV Javascript Ajax API provided in http://www.webdavsystem.com/ajax. But it also suffered the same issue.

Here is the piece of Javascript code to do the simple move.


function doResourceMove(from, to) {
var postParams = '<?xml version="1.0"?gt;<propertybehavior xmlns="DAV:"><keepalive>*</keepalive></propertybehavior>';
var xhr = getXHR(); //gets the HTTP Request object
xhr.open("MOVE",from, false);
xhr.setRequestHeader("Destination",to);
xhr.setRequestHeader("Overwrite","F");
xhr.setRequestHeader("Content-Type",'text/xml; charset="utf-8"');
xhr.send(postParams);

var status = xhr.status;
if(status == 201 status == 204 )
return true;
else {
throw status; //can be 201, 204, 403, 409, 412, 423, 502
}
}

After trying WebDAV Ajax API provided by webdavsystems, it was time for me to roll my sleeves up and analyze the issue.

The main issue was that the production system runs on https and there was a reverse proxy setup for this system. So, all the "https" requests were converted to "http" at this proxy level and forwarded to the main system. This was the main culprit.

Here is an example of HTTP request for the MOVE resource request for a WebDAV resource. For brevity I removed all unnecessary details.

MOVE contentLocation //request line, some https location, URI of webDAV resource
Destination:destinationLocation // this is the HTTP request header, should be absolute URI according to specifications.

Overwrite: "F" // this is also a HTTP request header

So, when the reverse proxy sees the request line, it knows that it has to convert this to HTTP request but the header Destionation also contains an HTTPS request which would be ignored by the proxy. So, when the request reached the server, we are moving the resource from an URI which begins with a http to a URI which begins with an https. Server treats this request as a request to move a WebDAV resource from one server location to another server location.(Refer RFC: http://greenbytes.de/tech/webdav/rfc2518.html#METHOD_MOVE). This was the source of the main problem.

Solution: When I changed the destination location to use the relative URL, it just worked fine. It may not be the solution for all WebDAV based systems but for me, it just did the trick.

Example:Let's assume that the resource you want to move is located at the URL location https://www.abc.com/webdavimpl/folder1/folder2 and you want to move this resource to location https://www.abc.com/webdavimpl/folder1/folder3/folder2. Following are the important HTTP request headers to follow.

MOVE https://www.abc.com/webdavimpl/folder1/folder2

Destination: /webdavimpl/folder1/folder3/folder2

Overwrite: "F".

1/12/2007

Customizing SAP Portal Master Header for Branding

SAP Enterprise Portal provides Theme Editor to edit the themes/styles for Portal environment. We can change the branding by changing the masthead part in the theme editor. But if you want to reposition the elements, add new elements or require more customizations then changing the code is a way to go. But before changing the code, it would be a better idea to check whether by editing the themes will suffice.

Changing the Code:

To change the masthead by changing the code, it would be a better idea to get the existing masthead par file and use it as a starting point. We can even develop whole new portal component using framework library and make it an iview out of it.

To get the mastead par file, go to System Administration -->Support --> Browse Deployment and browse to the directory ROOT/WEB-INF/deployment/temp and you will find com.sap.portal.navigation.masthead.par.bak file. Download it to your local system. If you observe the file extension, it is "bak", because SAP Portal renames the archives after it deployes the par so that it will not deploy it again. Remove the file extension .bak and rename according to the naming standards of your organization. I would recommend you to use the same naming of the file, remove "sap" and use your own company name, like com.mycompany.portal.navigation.masthead.par. After this, you can import this par file in NWDS (Netweaver Developer Studio) by selecting File --> Import. A word of caution, the importing does not import the jar files in the private/lib folder of the project structure. So, copy them manually to the new project you have created.

When you see the project structure, you can find the header related file in dist --> PORTAL-INF -->jsp --> Headerview.jsp. You can change this file according to your needs and deploy it back to the portal.

Create an iView:

In the portal environment, you can create iView out of this new portal component you just uploaded. To do this,go to Content Administration--> Portal Content --> Portal Content and I recommend creating a folder for placing all the customization related objects (iviews, framework page, portal desktop). To create an iview in the newly created folder, right click on the folder name and select "New iview from Par" (In Netweaver 2004s, select "new --> iview" and in the next step select the appropriate option for creating an iview from the portal component). Select the portal application (par file name) you just deployed to portal and select the portal component. It'll create the iview based on the portal component we just changed.


Framework Page:
For testing, we can create a portal framework page (just copy the standard portal framework page available at Portal Content -->Portal Users --> Standard Portal Users -->Default Framework Page ). Open the copied Framework page for editing and make visible checkbox unchecked for exisitng Mastehead iView and add the new masthead iview just created. Go to Page Layout and arrange the masthead so that it appears on the top.


Portal Desktop:
Now, copy the SAP provided standrd Portal Desktop object to the folder we created for placing the custom content. To do this, go to System Administration --> Portal Display -->Desktops & Display Rules and browse to Portal Content -->Content Provided by SAP --> End User Content -->Standard Portal Users and copy Default Portal Desktop and paste it in the your folder. Now, open the copied portal desktop object and add the newly created framework page and make it default framework page for this desktop object.


Changing Master Rule Collection:
Now, you can change the Master Rule Collection which controls the desktop selection for end users based on the criteria specified. To open the Master Rule Collection, go to System Administration --> Portal Display -->Desktops & Display Rules and browse to Portal Content -->Portal Administrators -->Super Administrators --> Master Rule Collection. Add a new rule for displaying the newly created desktop for some user in portal (testing purpose), save the object and close it.


Result:
You can now go and check the new portal desktop containing the new masthead you just created by logging into the portal as the user for whom you set the new rule.

8/17/2006

Using HTML Fieldset

Ajax is all about making websites more dynamic, richer and making them more usable. All the technologies that play part in Ajax are really important to achieve this goal. So, here in this article I explain the usage of fieldset and legend HTML elements, which add more semantic, structure and usability if used properly.

Fieldset is useful in making semantically related fields appear together visually. Grouping fields this way makes it easier for user to understand the semantic relationship of the fields and they are pretty stylish too, which makes it a very useful element.

Legend allows us to give a proper caption to the fieldset and makes it even easier for the user.

Let’s take a quick look at a simple example. Suppose that we are gathering personal and professional information of a user. One way of achieving this is to use markup elements (like, h1, h2, h3…) for these two sections. But using fieldset makes it very simple just by adding two fieldsets, like

<fieldset>
<!—for personal information -->
</fieldset>
<fieldset>
<!—for professional information -->
</fieldset>

This way, it adds semantic relationship among the fields declared inside a fieldset. To give the legend, just use element inside the fieldset, like

<fieldset>
<legend>Personal Information</legend>
for personal information
</fieldset>
<fieldset>
<legend>Professional Information</legend>
for professional information
</fieldset>

I added some styles to the fieldset and legend fields. Here is the result page:

http://pavankeely.brinkster.net/fieldset_example.html

We can style these elements the same way we style other elements using CSS. Change the border color, font, background color and much more.

Fieldsets can also be nested producing nested semantic relationship to the HTML document and will help the user to understand the relationship.

IE Specific Info: By default IE adds a rounded cornered border. But once you change the border style, width or color using styles, the rounded corners disappear.

7/06/2006

System error: -1072896658 in IE

Problem:
Recently my team reported a problem while integrating one of the Ajax features I developed. Whenever they try to access that feature, it throws up an error "System error: -1072896658". I was suprised because that is working fine when I access in my test environment. Later after debugging in the integration environment, I found out that the problem is with charset. While configuring the server environment, they mistakenly specified the charset as "UTF8" (instead of utf-8) and that is the culprit of this error. So, whenever the response is received in the browser and accessed using xhr.responseText, MSXMl.dll (MSXML component) is not able to interpret the response and throwing a system error.

Solution:
By changing the charset to "utf-8", it started working absolutely fine.

Conclusion:
Make sure to specify the correct charset.

7/05/2006

Closing Tab/Window in Mozilla using window.close

Category: workaround

Mozilla browsers will not allow you to close the window using window.close() unless the script opened the window. This is a little annoying sometimes. But there is a workaround to resolve this issue. If you observe the error message that is thrown by Mozilla Firefox, "Scripts may not close windows that were not opened by the script", it clearly says that if the script didn't open the window, you can't close that. But we open a blank page in the same window using "_self" as the target window and close the same window. In that way, the script opens the window (which is a blank one) and closes the window too.

So, to put this in the code,

< script >
function closeMe()
{
var win = window.open("","_self"); /* url = "" or "about:blank"; target="_self" */
win.close();
}
< /script >
< input type="button" name="CloseMe" value="Close Me" onclick="closeMe()" />

In mozilla, if there are more than one tabs open, then only the tab where this script is being executed, will be closed, not the whole window. But if there is only one tab open where this script is executing then the window will be closed.

Tested this script in Mozilla Firefox 1.5.0.4 and IE 6, XP SP2, Mozilla 1.7.12. I didn't test in any other browser.

5/30/2006

Using Dynamic Script Tags

Level: Beginner to Intermediate

One of the known problems in using XMLHttpRequest is that one can't make a request to a different domain than where the web page is coming from. It's a big limitation for some of the applications. There are some knows workarounds. For example, in IE we can change the security level by allowing access to data resources across domains. But we can't rely on this approach because not all other browsers support this configuration change.

There are other ways like using document.domain to set to a common domain, but this forces the response mechanism to HTML and in both the pages, we have to set the document.domain to same domain and this will not work if the domains are completely different. And many times, we may not have control over what's the response of the second domain request. One thing worth noting here is that subdomains are treated as separate domains.

If the response is script-centric then the best approach that can be used is "Dynamic Script Tagging". This appraoch doesn't use XHR. In this approach we construct script tags dynamically with the source (src attribute) pointing to the URL that has a response type as a script (script-centric response). Because the response is included in script tags, the response is evaluated by Javascript engine and that will be ready for use. I read some artciles describing this appraoch as JSON with Dynamic Script Tag. But I strongly oppose that. JSON is a pure data format. and if you just include pure JSON data as the response the data may be parsed correctly but has no real value. We need to capture that JSON data in some variable form or as a parameter to a function call. So, I call this method as Dynamic Script Tags with Script-centric approach ( It is not necessary to have JSON string included in the response, response can be a plain text assigned to a Javascript variable, like var test="this is a test" can be a valid response for this approach).

Let's see how we can use this method with an example. The example tries to get image search results from yahoo webservice API using two extra parameters output and callback.

output=json instructs yahoo webservice to send the response as JSON and callback will include method name in the response so that when Javscript evaluates the response, it knows that it has to call the method included in callback parameter. Example response for the yahoo webservice URL pointed by anchor:
Yahoo Image Search is

handleResults({"ResultSet":{
"totalResultsAvailable":"3",
"totalResultsReturned":2,
"firstResultPosition":1,"Result":[{"Title":"pKeely.jpg","Summary":"", "Url":"http:\/\/www.thepeoplephotographer.com\/hs01\/imgs\/pKeely.jpg",
"ClickUrl":"http:\/\/www.thepeoplephotographer.com\/hs01\/imgs\/pKeely.jpg",
"RefererUrl":"http:\/\/www.thepeoplephotographer.com\/hs01\/ahsagalleryjane.htm",
"FileSize":"19626",
"FileFormat":"jpeg",
"Height":"360",
"Width":"554",
"Thumbnail":{
"Url":"http:\/\/mud.mm-a6.yimg.com\/image\/2220417984",
"Height":"94",
"Width":"145"}
},{
"Title":"pkeely.jpg",
"Summary":"",
"Url":"http:\/\/www.qvcuk.com\/ukgif\/pkeely.jpg",
"ClickUrl":"http:\/\/www.qvcuk.com\/ukgif\/pkeely.jpg",
"RefererUrl":"http:\/\/www.qvcuk.com\/ukhtml\/elemis_expert.html",
"FileSize":"3722",
"FileFormat":"jpeg",
"Height":"100",
"Width":"100",
"Thumbnail":{
"Url":"http:\/\/mud.mm-a3.yimg.com\/image\/737339806",
"Height":"100",
"Width":"100"}}]}})

Working example link: http://www.geocities.com/keelypavan/dynamic_script_tag_ex.html

It constructs the script tag dynamically using document.createElement, sets the src attribute of the script tag and inserts that in head tag of the HTML page, simple....The code looks like this:

var headTag = document.getElementsByTagName("head").item(0);
var scriptTag = document.createElement("script");
scriptTag.src = url;
headTag.appendChild( scriptTag );

One problem I came across recently worth mentioning is, if the server request is using custom redirect then this method will not work. When I say custom redirect, I mean the responses using refresh attributes with META tags like:
<META HTTP-EQUIV="Refresh" CONTENT="0; URL=someURL">. In this case, Javascript can not evaluate META tags and it fails producing a Javascript error.

4/24/2006

Data Interchange formats for Ajax based applications

With the introduction of Ajax, the classic web-applications are moving towards sending data instead of content to the web browser (in most of the cases). The emphasis on data interchange formats is more than before. This article points out available data interchange formats.

If you consider a normal web-application (non-Ajax), server sends some content (normally, HTML content) and like a faithful servant, web browser displays it and it may have Javascript working but to a limited level. But when we talk about richness of the application, we need something more than this. Most part of how we display the content should be left to the application running in browser, so that it can change the content or even look and feel dynamically, i.e. Ajax app, especially Javascript.

As we all know, the XMLHttpRequest is the core component of Ajax and it communicates with the server to get data to display in the browser without any reload of the page. Different applications use different data formats based on their application needs.

Following types of data interchange I can think of in the industry now.

- XML (eXtensible Markup Language)
- JSON (Javascript Object Notation)
- String with delimiters
- Script-centric approach
- Classic way of sending content.


Well, first three formats together can be considered data-centric approaches. But for clarity I am separating them. Let’s discuss each of the formats individually.

XML:

XML is a web standard for data interchange formats. It’s been around for quite sometime now. The support for XML in Javascript is very good as most of the browser implemented XML DOM specifications. The main usage of XML is that structured and hierarchical data can be represented very well and it’s readable by human beings. This comes with a cost of including meta-data, which describes what that data represents. Of course, I have seen many XML documents, which you can’t make out anything from but let’s keep that aside for now. One good thing about this is its pure data representation, which lacks in HTML. (HTML represents data intermingled with styles and formatting.)

Once you get the XML content as a response to the client-side you can use XSLT to transform the data into HTML content or you can use XML DOM API to parse and access XML and form HTML content, may be using innerHTML or standard DOM API.

Let’s take an example and see how we can represent the same data/content in all the formats. The example I am going to take is folder contents’ information. The XML is self-explanatory, so I am not spending much time explaining what it represents.


<?xml version="1.0"?>
<items>
<item>
<name>Test Document</name>
<type>document</type>
<creator>Test creator</creator>
</item>
<item>
<name>Test Folder</name>
<type>folder</type>
<creator>Test Creator 2</creator>
</item>
<item>
<name>Test Shortcut</name>
<type>shortcut</type>
<creator>Test Creator 3</creator>
</item>
</items>

JSON:

JSON is a light-weight data interchange format. It’s a text (string) representation of Javascript objects. An object in Javascript is represented in key, value pairs. A key is a string enclosed in double-quotes whereas the value can be number, string, boolean (true or false), object, array or null. Following paragraph explains JSON format in brief.

  • An object is a set of name, value pairs and it’s enclosed in "{" and "}". Name and value is separated by " : " and Name, value pairs are separated by " , ".

  • An array is ordered collection of values and is enclosed in "[" and "]" and values are separated by " , ".

  • Name is a string enclosed in double-quotes.

  • Value can be anyone of the following. String, Number, Boolean (true or false), Object, Array, null.


  • The advantage of JSON is that it’s more compact than XML format and parsing JSON is a lot simpler than XML. You just need to pass JSON string to eval of Javascript or you can also download JSON parsers for different programming languages from http://www.json.org/. As we have parsers for most of the famous programming languages, it makes JSON a good data interchange format. I know that a lot of people think that eval is very evil but doing eval once to evaluate the JSON string will not cause any big performance impact. But if the data grows larger then definitely JSON will not be a good option.

    Example: Let’s take the same example I represented in XML and write that in JSON format.

    {items:[
    {"name": "Test Document", "type": "document", "creator": "Test creator"},
    {"name": "Test Folder", "type": "folder", "creator": "Test Creator 2"},
    {"name": "Test Shortcut", "type": "shortcut", "creator": "Test Creator 3"}
    ]}

    String with delimiters:

    Though it’s not a standard to use a plain string with delimiters as the response format, it has some advantages. We can use regular expressions or split the string into pieces using the delimiter and use the data as an array. Not much of processing is required for parsing the string.

    The problem with this approach is that we need to rely on the position of the elements. And if there is any change in the positions of the elements (data) then it requires a change in the client side code. And representing deep hierarchical data is very difficult and error-prone in this approach.

    Example:
    The data represented in two examples above could be represented in this approach as:

    Test Document#document#Test creator$$Test Folder#folder#Test Creator 2$$Test Shortcut#shortcut#Test Creator 3

    As you see this format is very compact because it doesn’t contain any meta-data but as the nested ness of data increases, like corporate taxonomy, representing that data is this fashion would definitely be a problem.

    Script-centric approach:
    In this approach a piece of script will sent from the server like assigning values to variables, function calls, which will be dynamically evaluated at the client side to perform the necessary actions.

    The disadvantage of this approach is that it assumes some Javascript environment (like some functions defined in the page) at the client side. This means more coupling with the response with the page that’s requesting the resource.

    Example: As we can’t represent data as-is and there will be piece of Javascript code as a response in this approach, there could be multiple ways you can represent this.

    Response:

    var matchingItems = {items:[
    {"name": "Test Document", "type": "document", "creator": "Test creator"},
    {"name": "Test Folder", "type": "folder", "creator": "Test Creator 2"},
    {"name": "Test Shortcut", "type": "shortcut", "creator": "Test Creator 3"}
    ]} //new lines are just for clarity.

    someMethod( matchingitems,… )


    Content-Centric Approaches:

    In the content-centric approach, the response from the server contains HTML content. So, the client side application (Javascript) has to take the content as is and place the content in any container using innerHTML or related methods. The advantage of this approach is that there is no explicit parsing of the data required. I used “explicit”, because internally when you use HTML content, the data has to parsed and shown in the web page. But the disadvantage is that it’s data with formatting tags. If you want to reuse the data of a response in this approach then we have to rely on DOM methods to retrieve the data, which is cumbersome.

    Example:

    As the response would be in HTML format, we can represent this in many ways depending on the need. Here is a way:

    Response:


    <div><span>Test Document</span><span>document</span><span>Test Creator</span></div>
    <div><span>Test Folder</span><span>folder</span><span>Test Creator2</span></div>
    <div><span>Test Shortcut</span><span>shortcut</span><span>Test Creator3</span></div>


    We can take this content as-is and insert as HTML content in any of the allowed elements using innerHTML (or related) method. There’s not much processing required at the client side.

    Sites/Apps using these formats:

    XML:

  • Netflix

  • Dell

  • Google suggestions toolbar (new beta)
  • Script-centric approach:

  • Google Suggest

  • JSON format:

  • Yahoo provides JSON for its web services.
  • Conclusion: Depending on application needs, appropriate data interchange format has to be chosen. This article just describes the options available and the decision is yours.

    I’ll update this article whenever I get some more time.