Thursday, August 27, 2009

XQuery Rename/Replace Xml Node and Update K2 Workflow Process Instance Xml Fields

I recently ran across a case where a property in our application was re-termed due to a change in requirements. Updating the code was very easy, however it meant that all of the Xml documents needed to be updated and that was not going to be as easy. It would involve updating the serialized object xml documents in our workflow data repository and K2.

Out of curiosity I ran the process using the old Xml to see what would happen. Deserializing the old Xml documents resulted in everything after that renamed property being omitted from the deserialization and it didn't throw and error. That was unexpected, but good information to know.

As it turns out, updating the workflow data repository was easy. K2 has many issues which resulted in the K2 update sql. The code below shows how I accomplished this:
CREATE PROCEDURE [dbo].[usp_brock_RenameXmlNode]
AS
BEGIN
 -- UPDATE Workflow Instances
 UPDATE [WorkflowInstances] SET InstanceData.modify('
  insert element NewTag {/ObjectXML/OldTag/text()} after (/ObjectXML/OldTag)[1]
 ')
 WHERE  WorkflowID='7B4DEB62-D462-4FE8-A8E2-2057B5F31B19'
   AND InstanceData.exist('/ObjectXML/OldTag') = 1
 UPDATE [WorkflowInstances] SET InstanceData.modify('
  delete (//OldTag)[1]
 ')
 WHERE  WorkflowID='7B4DEB62-D462-4FE8-A8E2-2057B5F31B19'
   AND InstanceData.exist('/ObjectXML/OldTag') = 1
 
 -- UPDATE K2 Workflows
 -- Create Table Var and Populate
 DECLARE @K2ProcXml2XmlType table
 (
  ProcInstID INT PRIMARY KEY,
  WorkflowName VARCHAR(128),
  WorkflowID UNIQUEIDENTIFIER,
  InstanceData XML,
  IsModified BIT
 )
 PRINT 'POPULATE TABLE VAR';
 INSERT INTO @K2ProcXml2XmlType 
  SELECT pin.ID
     ,ps.[Name] WorkflowName
     , wi.WorkflowID
     ,CONVERT(XML,REPLACE(CAST(f.string as nvarchar(max)),'<?xml version="1.0" encoding="utf-8"?>','')) InstanceData
     , 0 --IsModified
    FROM [K2].[dbo]._FieldOnDemand f INNER JOIN [K2].[dbo]._ProcInst pin ON f.ProcInstID=pin.ID
    INNER JOIN [DemoDatabase].dbo.WorkflowInstances wi ON wi.ProcessID=pin.ID
    INNER JOIN [K2].[dbo]._Proc p ON pin.ProcID = p.ID
    INNER JOIN [K2].[dbo]._ProcSet ps ON p.ProcSetID = ps.ID
  WHERE  (ps.[Name] in ('WorkflowX'))

 UPDATE @K2ProcXml2XmlType
 SET InstanceData.modify('insert element NewTag {/ObjectXML/OldTag/text()} after (/ObjectXML/OldTag)[1]'), IsModified = 1
 WHERE (InstanceData IS NOT NULL) 
  AND (InstanceData.exist('/ObjectXML/OldTag') = 1)

 UPDATE @K2ProcXml2XmlType
 SET InstanceData.modify('delete (//OldTag)[1]'), IsModified = 1
 WHERE (InstanceData IS NOT NULL) 
  AND (InstanceData.exist('/ObjectXML/OldTag') = 1)


 -- End Result of all updates
 --PRINT 'QUERY RESULT:';
 --SELECT
 --        [ProcInstID]
 --   , WorkflowName
 --   , InstanceData
 --  FROM @K2ProcXml2XmlType src
 --WHERE src.IsModified = 1;

 -- Populate Original Table and Query
 PRINT 'POPULATE TABLE FROM VAR:';
 UPDATE dst
 SET dst.String = CONVERT(ntext,'<?xml version="1.0" encoding="utf-8"?>'+CAST(src.InstanceData as nvarchar(max)))
 FROM @K2ProcXml2XmlType src INNER JOIN [K2].[dbo].[_FieldOnDemand] dst ON src.ProcInstID = dst.ProcInstID
 WHERE src.IsModified = 1;

        -- END K2 Update
END;
GO

exec dbo.usp_brock_RenameXmlNode

DROP PROCEDURE dbo.usp_brock_RenameXmlNode
It runs quite fast.

There are a couple caveats, you can't create the XPath location dynamically (unless you are using dynamic sql which I try to avoid at all costs). This means that you can't pass old and new name parameters into the stored procedure and update the records. Disappointing as it is, any future renames should require an easy find and replace.

I tried using a cursor to do this type of updating, but the resulting SQL in execution times in more than more than 90 minutes (I cancelled it at that point) and verified with the DBA's that I was not doing something wrong, I abandoned that path and went back to the set based updates.

Wednesday, August 26, 2009

Visual Studio 2008 Post-Build Event and Minify Using YUI Compressor

A time tested way to shrink and optimize JavaScript and CSS is to minify and obfuscate your files. Yahoo (High Performance Web Sites Rule 10) and Google (Minimize Payload Size) have best practices recommending minifying JavaScript.

The YUI Compressor is an excellent open source tool for this; the best I have found. My real world use reduced the size of the regular CSS and JavaScript files by 45%. Gzipping the files reduced CSS files by an additional 25% and JavaScript files by an additional 36%. Resulting in 75%+ smaller files.

Here is a batch file snippet which should speed up implementing this into your build process:

SET YUICOMPRESSOR=%Minify%\yuicompressor-2.4.2.jar
IF "%JAVA_HOME%" == "" (
 ECHO Searching for Java...
 for /f %%j in ("java.exe") do (set JAVA_HOME=%%~dp$PATH:j)
)
ECHO Java: %JAVA_HOME%
ECHO YUICOMPRESSOR: %YUICOMPRESSOR%
IF EXIST "%YUICOMPRESSOR%" (
 IF NOT "%JAVA_HOME%" == "" (
  ECHO Deleting Minified files, TFS can make these readonly and the compressor will not overwrite read only files...
  del /F "%CD%\<Path To Script Directory>\core.min.js"
  del /F "%CD%\<Path To Styles Directory>\core.min.css"
  ECHO Done.

  "%JAVA_HOME%java" -jar "%YUICOMPRESSOR%" -v --type JS -o "%CD%\<Path To Script Directory>\core.min.js" "%CD%\<Path To Script Directory>\core.js"
  "%JAVA_HOME%java" -jar "%YUICOMPRESSOR%" -v --type CSS -o "%CD%\<Path To Styles Directory>\core.min.css" "%CD%\<Path To Styles Directory>\core.css"
 ) ELSE (
  ECHO ERROR: Java is not installed, no Minification
 )
) ELSE (
 ECHO ERROR: YUI Compressor not found.
)

Also note that if a build event echos a line starting with "ERROR", not sure if it is case sensitive, the Visual Studio and Team Foundation Server will set that step as a failure and alert as expected. Remove the "ERROR:" text from the echo lines to allow the build to proceed if it cannot find the compressor or java.

jQuery Predicate Selectors and ASP.NET

Well, I finally get to blog on something that I get to use in my every day life. There is not much information on the internet regarding to jQuery predicate selectors, so I figured I would blog about how I used them.

The need came up when I was trying to move JavaScript into an external file. The code was put on the page because it referenced ASP.NET controls on the page requiring the use of the old dynamic JavaScript/ASP.NET client ID injection pattern. It seemed like jQuery should be able to find the controls given the last part of the ID. After many google searches and changing of terminologies (I can't remember the search that led me to the solution), I ran across a post from Ben Nadel titled Cool jQuery Predicate Selectors. He describes exactly what I was looking for and more:
Start With: ^=
The ^= operator will filter elements whose attribute starts with the given value.

Ends With: $=
The $= operator will filter elements whose attribute ends with the given value.

Contains: *=
The *= operator will filter elements whose attribute contains the given value.

He goes on to demonstrate how to use these tools, but doesn't go into how it can be used in ASP.NET; that's where I pick up.

Using basic JavaScript to reference an ASP.NET control you have to put a block of code in the aspx page. For example:

<script>
aspnetControl = document.getElementById("≶%= serverControl.ClientID %>");
</script>

This is all well and good until there are a large number of ClientID's in your page. They are generally very long and can bloat the size of your rendered page. We can use the predicate selectors to improve this case. Lets assume the ClientID ends with "serverControl", per the example above, and that it renders an html input tag.

<script>
aspnetControl = $("input[id$='serverControl']");
</script>

Short and sweet. This can be refactored and placed into an external file which can be cached.

As for performance, it is safe to assume that the document.getElementById call would be fastest followed by using the jQuery ID selector (which uses the getElementById command) then the predicate selectors. In my real world use of predicate selectors, it seems to be as fast or faster than the the other methods using IE 8 and Firefox 3.0/3.5.

This can go a long way to decreasing the page size and loading/rendering speed of the page. According to Google's Page Speed Best Practices:
You should see a benefit for any file that can be reduced by 25 bytes or more (less than this will not result in any appreciable performance gain).

Wednesday, August 19, 2009

SQL Server 2005 Extract Bitwise Flags

UPDATE: 8/20/2009 - This post has been updated several times, so read through for all of the information. I ran into a scenario where I had a numeric field which was composed of flags and I needed to extract them. I was told that you couldn't do bitwise (AND/OR) operations, but some undocumented bitwise system functions existed (like fn_IsBitSetInBitmask). Trying to dig into the system function was not very easy and having no real documentation outside of the "exec sp_helptext 'fn_IsBitSetInBitmask'" query, I wasn't very successful.

I started to have memories from my Digital Logic class where we had to do something similar using modulus with a constrained feedback loop. I pseudo-coded an example function, but found that SQL doesn't support arrays (thinking now, I probably could have used a table variable, but that would have needlessly complicated the situation). Regardless of SQL's deficiencies, using the modulus/feedback method does work, but wasn't going to fit my vision of the solution.

function BitFlags(int val) {
 bool[] arr = new bool[8];
 int modulus;
 for(int i=7; i >= 0; i--) {
  int pwr = (2^i);
  if (val == pwr) {
   arr[i] = true;
  } else if (val > pwr ){
   modulus = val % (2 ^ i);
   if (modulus != 0) {
    arr[i] = true;
    val = modulus;
   }
  }
 }
 return arr;
}

For those who haven't used their math skills in a while, here is an example. My Digital Logic skills are a bit rusty, so I may be incorrect, feel free to correct me if I am wrong. We want to find the bit flags for the number 76; using the modulus/feedback method, we get something like:

// Background
2^7 = 128
2^6 = 64
2^5 = 32
2^4 = 16
2^3 = 8
2^2 = 4
2^1 = 2
2^0 = 1

// Now the fun part, cycle through the range 0-7 for 2^i:
// Calc         Output    Description
76 % 128 = 0    [0]    // Feedback 76
76 % 64  = 12   [1]    // Feedback 12
12 % 32  = --   [0]    // 12 is smaller than 32, Feedback 12
12 % 16  = --   [0]    // 12 is smaller than 16, Feedback 12
12 % 8   = 4    [1]    // Feedback 4
4  % 4   = --   [1]    // 4 == 4, so the we get a 1 for the output and there is nothing to feedback, so everything past that point is 0
0  % 2   = --   [0]    // 0 is smaller than 2, Feedback 0
0  % 2   = --   [0]    // 0 is smaller than 2, Feedback 0

Something just did not settle well with me. You just had to be able to do bitwise operations; it seems like elementary funcitonality. I took my C skills and tried to them out in SQL. It threw me for a loop when after evaluating the @val & <number> expression that I would get back the <number> where I was expecting a 1, but 0's everywhere else. I new I was on the write path and was almost there. I did some googling and found:

http://www.sqlusa.com/articles2005/binarypattern/

Which was doing something similar, but didn't exactly fit the situation. It gave me a DUH! moment when I realized that I should just divide by the <number> and POOF, I had the elegant solution I was expecting.

Try it out for your self.
DECLARE @val as tinyint
SET @val = 128+64+16+1

SELECT  (@val&128)/128
      , (@val&64)/64
      , (@val&32)/32
      , (@val&16)/16
      , (@val&8)/8
      , (@val&4)/4
      , (@val&2)/2
      , (@val&1)/1

--RESULT:
--11010001

UPDATE: My good friend Christopher Lauer used my code and showed me how he used it. I did not find anything like this when I googled and thinking that it would be useful for other people he gave me permission to include it.

USE MSDB;

if not exists (select * from sys.schemas where name = 'my_utility')
 exec('create schema my_utility')
go

/*****************************************************************************************
 DESCRIPTION: The MSDB.dbo.sysSchedules has a column named freq_interval, this 
 column may contain a packed bit that needs unpacked to understand what days
 the schedule is set to run on.  See books on line for information on this
 table and column.  This function will unpack the bit and return a string 
 of weekday names that the bit respresents.

 Base code provided by Brock Moeller
 --SELECT [Monday] = (@bitWise&2)/2, [Tuesday] = (@bitWise&4)/4, [Wednesday]= (@bitWise&8)/8, [Thursday] = (@bitWise&16)/16, [Friday] = (@bitWise&32)/32, [Saturday] = (@bitWise&64)/64, [Sunday] = (@bitWise&128)/128

 MAINTENANCE: 
 [date:name]      [description of the change/maintenance being done]
 ==============================  ==================================================
 8/19/2009: Christopher M. Lauer  Created function.
******************************************************************************************/
CREATE FUNCTION [my_utility].[udf_getDayNameFromBitWise](@bitWise AS INTEGER)
RETURNS Varchar(62) AS
BEGIN
 --SET NOCOUNT ON;
 /*
 DECLARE @bitWise INT;
 --SET @bitWise = 63; --Monday, Tuesday, Wednesday, Thursday, Friday
 */

 /***** LOCALS ******/
 DECLARE @ErrorCode INT, @RowsAffected INT;
 SELECT @ErrorCode = 0, @RowsAffected = 0; -- default
 /***** END LOCALS ******/

 DECLARE @WeekTable TABLE(dName VARCHAR(10), isUsed bit, dNumber tinyint, bitWise int)
 INSERT INTO @WeekTable(dName, isUsed, dNumber, bitWise)
 SELECT 'Monday', (@bitWise&2)/2, 1, 2
 UNION ALL 
 SELECT 'Tuesday', (@bitWise&4)/4, 2, 4
 UNION ALL 
 SELECT 'Wednesday', (@bitWise&8)/8, 3, 8
 UNION ALL 
 SELECT 'Thursday', (@bitWise&16)/16, 4, 16
 UNION ALL 
 SELECT 'Friday', (@bitWise&32)/32, 5, 32
 UNION ALL 
 SELECT 'Saturday', (@bitWise&64)/64, 6, 64
 UNION ALL 
 SELECT 'Sunday', (@bitWise&128)/128, 7, 128
 --SELECT * FROM @WeekTable

 DECLARE @ReturnValue as VARCHAR(50)
 SELECT @ReturnValue = COALESCE(@ReturnValue, '') + dName + '; ' 
 FROM @WeekTable
 WHERE isUsed = 1;
 
 IF(LEN(@ReturnValue) > 1) 
    begin
   SET @ReturnValue = substring(@ReturnValue,1,len(@ReturnValue) -1)
    end
 RETURN @ReturnValue
END
UPDATE: 8/20/2009 - I had a little free time and discovered an odd occurrence with bitwise operations.
DECLARE @val as tinyint
SET @val = 128+64+16+1
SELECT (@val&10000001)
--RESULT 129
SELECT (@val&01000001)
--RESULT 65

SELECT (@val&00100000) -- @val&32
--RESULT 128  ---- Incorrect!  Should be '0'.  
It appears that SQLServer 2005 implicitly interprets the number as binary. I can't find any documentation on this issue. The results are accurate for scenarios where what you are looking for matches. However, if you test on something that doesn't match the left value, it doesn't return the correct number. I suppose an answer will have to wait.

On a side note, hexadecimal numbers work as expected like their decimal counterparts.
DECLARE @val as tinyint
SET @val = 128+64+16+1
SELECT (@val&0x40)
--RESULT 64

SELECT (@val&0x20)
--RESULT 0
It still appears that sticking with base 10 is easiest, but if you are good with hex you may find that usefull.

Friday, August 7, 2009

It's MSDN, but More Usable

So, I started writting this blog back in April and completely forgot about it.  Better to post late than never I suppose.

I use the MSDN site on occasion and have always found it rather wanting.  Not because of a lack of content, but the speed of the site and the small content view caused by the large header (which you previously could not get rid of without some GreaseMonkey magic).

It appears that people are listening and there are some easter eggs that you can play around with.  Scott Hanselman has a great write-up on the easter eggs and Jon Galloway has a more focused post.  Apparently some of this information came out late last summer, but I had not heard of it until a couple months ago and it's never too late to spread the information.

There are 3 easter eggs that I am a huge fan of, namely the Low Bandwidth view, PDA view, and the Visual Studio 2010 view. 

To get at the easter eggs, it requires some URL magic.  The MSDN developers use a form of URL routing (read Scott's post if you want a more information) using the contents of the parathesises as switches.  You may have seen the "(printer)" prefixed to the "printer friendly" view before.  This uses the same concepts.

My favorite is the "Low Bandwidth" view: (loband)

You can test it out by checking out: http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.spsolution(loband).aspx

The "PDA" version (http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.spsolution(pda).aspx) is very nice when you just want the information and just the information.  The Visual Studio 2010 version (http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.spsolution(dev10ide).aspx) will probably be my favorite in a little while... well it would be if it still worked.  Perhaps Microsoft is pulling a trick from the TV studio's and getting read of the good stuff.  At any rate, at least they the loband and pda version's still work.