Discussion Locked

Naren Cool Geek - Tips on .NET and Java development.

0 helpful
320 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

This is page 3 of the discussion. Read the opening post

320 replies

by narendrn

I have often wondered how download managers worked? I wanted to know the internal working that makes it possible for these components to download faster?

The main functions of a download manager are:
? Resuming interrupted downloads (i.e., downloading only the rest of the file instead of restarting theprocess from the very beginning);
? Scheduled operation: connecting to the Internet, downloading a list of specific files and disconnectingaccording to a user-defined schedule (e.g. at night when the connection quality is usually higher, while the connection rates are lower);
? some download managers have additional functions: searching for files on WWW and FTP servers byname, downloading files in several "streams" from one or from different mirror servers, etc.

Most download managers use the concept of "Multi-connection downloading" - the file is downloaded in several segments through multiple connections and reassembled at the user's PC.
To understand how this would work, we first need to understand a feature of web-servers.
A lot of webservers (http and ftp) today support the "resume download" function - what this means is that if your download is interrupted or stopped, U can resume downloading the file from where U left it. But the question now arises, how does the client(web-browser) tell the server what part of the file it wants or where to resume download? Is this a standard or server proprietary? I was suprised when I found out that it is the HTTP protocol itself that has support for "range downloads", i.e. when U request for a resource, U can also request what portion/segment of the resource U want to download. This information is passed from the client as a HTTP header:

See http header snipper below:

GET http://lrc.aiha.com/English/Training/Dldmgrs-Eng.pdf?Cache HTTP/1.1
Host: lrc.aiha.com
Accept: */*
User-Agent: DA 7.0
Proxy-Authorization: Basic bmFyZW5kcjpuYXJlbjEyNDM=
Connection: Close
Range: bytes=0-96143

Now what download managers do is that they start a number of threads that download different portions of the resource. So the download manager will make another request with the header:

GET http://lrc.aiha.com/English/Training/Dldmgrs-Eng.pdf?Cache HTTP/1.1
Host: lrc.aiha.com
Accept: */*
User-Agent: DA 7.0
Proxy-Authorization: Basic bmFyZW5kcjpuYXJlbjEyNDM=
Connection: Close
Range: bytes=96143-192286

This solves the mystery of how the download managers are able to simultaneously download different portions of the resource.

Imp Note: To resume interrupted downloads, it is not enough to use a download manager: the server from which the file is being downloaded should support download resumption. Unfortunately, some servers do not support thisfunction, and are called ?non-resumable.?
So Ur download managers won't work (no increase in speed either), as the servers would ignore the HTTP "range" header.

But I was still confused how exactly does this increase the speed. After all if the current bandwidth is "fully utilized" with one connection, how does making more connections help? The answers I found on the net are as below:

Normal TCP connections, as used by HTTP, encounter a maximum connection throughput well below that of the available bandwidth in circumstances with even moderate amounts of packet loss and signal latency. (so bcoz of packet loss and latency, the client will have to re-request some packets ) Multiple TCP connections can help to alleviate these effects and, in doing so, provide faster downloads and better utilization of available bandwidth.

Opening more connections means less sharing with others. Web servers are set up to split their bandwidth into several streams to support as many users downloading as possible. As an example, if the download manager created eight connections to the server, the server thinks it is transmitting to eight different users and delivers all eight streams to the same user. Each of the eight requests asks for data starting at a different location in the file.

Here are the links to some good download managers:

www.getright.com

www.internetdownloadmanager.com/

www.netants.com

www.alwaysfreeware.co.uk/dload.html

This post originally appeared on an external website

by narendrn

I came accross this slang many a times over the last few years. So what does "eating Ur own dog food " mean?

?A company that eats its own dog food sends the message that it
considers its own products the best on the market.?

?This slang was popularized during the dotcom craze when some
companies did not use their own products and thus could "not even eat
their own dog food". An example would've been a software company that
created operating systems but used its competitor's software on its
corporate computers.?

This post originally appeared on an external website

by narendrn

I often used to get frustrated with the speed of eclipse on my machine. Then I heard from my friend how I can increase the initial memory allocated to Eclipse. U just have to pass values from the command prompt using the vmargs argument.

eclipse -vmargs -Xms256m -Xmx256m

This single change, gave me a very good "perceived" performance benifit while using the IDE.

Other tips to increase the performance of Eclipse can be found at:
http://eclipsewiki.editme.com/GeneralFaq

This post originally appeared on an external website

by narendrn

In Java, when a class is declared as 'protected', it is also given 'package' access automatically. i.e. a protected member can be accessed within the same package by other members.

But in .NET, 'protected' access specifier means that only subclasses can see it. If we want other members of a assembly to see the 'protected' class, then we need to put the access specifier as 'protected internal'.

Thus 'protected' in Java is equivalent to 'protected internal' in .NET

This post originally appeared on an external website

by narendrn

There are a couple of things we need to keep in mind while doing a file-upload in ASP.NET.

If the upload fails it could be bcoz of the following reasons:

ASP.NET limits the size of file uploads for security purposes. The default size is 4 MB. This can be changed by modifying the maxRequestLength attribute of Machine.config's element.

If we are using the HtmlInputFile html control, then we need to set the the Enctype property of the HtmlForm to "multipart/form-data" for this control to work properly, or we might get 'null' for the PostedFile property.

This post originally appeared on an external website

by narendrn

At first I thought that the "package" access-specifier in Java is equivalent to the "internal" access-specifier in .NET. But actually there is a subtle difference.

The 'package' access-specifier in Java limits the access to a particular package/namespace, where as the 'internal' access-specifier in .NET limits the access to the containing assembly. Now a .NET assembly can contain more than one 'namespace' or package.
So is there any way to restrict the scope of a member of a class to the same namespace?

Suprisingly NO, there is none in .NET..So if we were designing by 'separation of concerns', maybe it would make sense to map one namespace to one physical assembly.

This post originally appeared on an external website

by narendrn

I have been a strong advocate of Struts right from the start. I often see developers getting confused over making a choice between struts, velocity, tiles etc.

The point is that "Struts" is a "controller-centric" MVC solution. The "View" and the "Model" parts are pluggable thanks to the plug-in architecture of Struts1.1
Hence is possible to use "Tiles" or Velocity templates inplace of plain JSP for the View in Struts.
Similarly the Model can be used to interact with EJB, JDO, Hibernate...etc.

The various choices available in Server-side Java may seem daunting to a new developer,but with experience we come to realize and appreciate the value of each solution and where it fits best :)

This post originally appeared on an external website

by shimonbenamy

I started with struts when i was in the last year of my developper studies. I saw that with a big application you can work properly. I like very well the mvc model, i think it s good way to develop web application.

we are never speaking about the opensource project in the time. Struts now is a ex jakarta project without any support, anyn new version.

So , i'm loving struts but how you can provide a web application based on the struts model but you 're not sure that you can working with struts for a long time(more than 2 or 3 years).

You can migrate to the JSF but when you have big apllication it's very dangerous.

But the developper are living dangerously ;-)

When an opensource project close , you have a big problem.

See ya

Shimon@bxl

by jrmint

I'm a Struts developer but since there hasn't been any move in a while
since 1.2 and the original Number 1 and Number 2 committers on the
Struts project are now backing JSF implementations in place of Struts
that looks like a product with a longer lifespan. JSF is MVC just like
Struts and can use the Validator in the same way as Struts so why limit
myself to a Web Server with Struts when with a simple change to an xml
file I can have a JSF app run standalone as an applet, web site, in a
portlet in a portal or be readable with handhelds?

by porky625

I would like to try it.

Any good tutorial which can helps to follow?

And by the way, which IDE will use for structs? any recommendation?

by johnnysacks

I got involved with struts after reviewing the initial example of a controller servlet detailed in (as I remember) the book Core Servlets And Java Server Pages.  After using the example in the book for building a tutorial project for a group of developers transitioning from ASP to Java, struts was a logical next step towards a more refined web application framework.

The reason that it's no longer in the spotlight as an open source project may be because it has reached the limit of the functionality it was initially designed to provide.  It performs a job, it performs it extremely well, and it will continue to do so.

by wayne m.

As a manager, my view on Struts is more pragmatic.  Struts is well defined and there is quite a bit of literature, both on the internet and in bookstores, to describe it.  For this reason alone, I endorse its use in projects.

From a purely personal evaluation, and at the risk of being labeled a fire-starter, though, I find Struts to be overly complex and largely applies MVC solely for the sake of doing MVC.  A developer needs to learn JSP and XML and perform indirect mapping of actions.  JSP code does not promote code reuse thoughout the project and changes and corrections to data fields need to be propogated manually across many pages.  This is an open invitation to errors when fields or pages are overlooked.  Similar issues ocurr on the database side, though the database tends to be more static.

I am not a big fan of the MVC pattern and feel few projects actually benefit from the independence of the three layers.  I find much of the purported benefit to be illusory as the differences between a stateful client-server design and a stateless web-based design affect the entire software.  One cannot simply skim off the View and change a client-server to a web-based system.

Putting my personal opinions aside, however, the reality is that Struts is a defined approach and I am much more likely to find personnel and implement a project successfully using Struts than a custom approach.

 

by Member 35274876

Struts is still alive an kicking, if you look at sites like HP support, Airlingus.com etc you'll see its used
for business applications

Personally I use M7 for Struts development, they have just been bought by Bea so as regards
getting the job done its just that.

We should always look to the future and new framworks but companies that have invested
in applications like to get their moneys worth. I expect Struts will be around for some time
as its a quiry bug mostly bug free framework.

It great to see that JSF is being influenced by the likes of the Struts creators, for me JSF will be looked
as when developing new greenfield projects. For the others, well the run reliably and if the bosses want to
change the framework ( dont think they will pay the money ) then I'll start strutting to JSF.

by naveen_arur

Struts now is a ex jakarta project without any support, any new version.
Can somebody please confirm this please?
I am new to java web development and I thought of putting some struts
skills into my tookit but if struts is no longer under development I am
not sure i want to continue my struts learning an dinstead jump on to
the jsf bandwagon

by the chad

Struts is absolutely a live project, however it--like
all the other Apache projects--has community support. Apache does not
support their products (there are some companies that may provide
support, though).

Even if struts died tomorrow, who cares? We have the source and could easily extend it and continue to fix bugs. That's THE POINT OF OPEN-SOURCE SOFTWARE, unlike closed source software. Once the company goes, that's it for you. For example, there is a very nice application called Powertab editor.
Even though it is free to use, it is not open-source. It hasn't been
updated in over three years or so, so it appears to be a dead product.
There is no way to fix bugs, add new features, etc., so everyone who
uses it must either live with its shortcomings or find something else.

I agree that not all projects require Model II archetecture (MVC).
However, for anything that starts out small with the potential to grow,
Model II is absolutely essential. Yes, it takes a little longer to
write, but the key is maintenance. Model I is an absolute bear to maintain.

For those of you who think struts is way too hard (hand editing the
config files and manually creating all the form getter/setter files, no doubt), you're doing way too much work. Grab yourself
a copy of Struts console and Eclipse, and you'll find these tools will automatically generate much of the mundane code one has to work with when using Struts.

Cheers

by atan

We wrote an application in Apache Struts with no prior knowledge of the
framework. 3 months down the track and it was such a joy to use. And as
a bonus, at the end of the project we'd learnt more about MVC than any
textbook could possible impart. Sweet...

We also used MyEclipse, Tomcat, MySQL, Hibernate, JFreeChart, Maven, iText, JUnit and DbUnit.

by barddzen

I agree somewhat that Struts is overly complex for small projects. 

Example:  Can Struts be used for a simple online form? 
Yes.  The bigger question is:  Is Struts the right solution
for this particular task?  IMHO, no, it's not.  Setting up
all the XML, Actions, Mappings, Validators, etc. just to capture 10
fields on a web site and send an email is over complicating something
that you could have coded in Straight JSP in less than 1 hour.

Now, having said this, would I consider Struts for larger "web applications"?  Absolutely.

Which is where I personally draw the line between using a framework
like Struts or any other framework.  Web applications are a far
cry from simple online capture forms and keeping it simple is ALWAYS
the best approach.

by narendrn

The .NET API has quite a few classes to help us manipulate XML data.
We have the 'XMLReader' class and its subclasses like 'XMLTextReader', 'XMLValidatingReader' ect that provide a forward-only, non-cached, read-only cursor to XML data. An interesting point is that XMLReader uses a 'pull model' unlike SAX push event model.

Then we have XPathDocument that provides a fast, read-only cache for XML document processing using XSLT. Quite good if U need to use XPath to query the XML data.

We have a special class for loading a DataSet directly into a XML form for manipultion. This class is the XMLDataDocument class that can load either relational data or XML data and manipulate that data using the W3C Document Object Model (DOM).

This post originally appeared on an external website

by narendrn

Often in my applications, I needed to persist some Java data objects as XML. For this I used to use JDOM which was quite cool to use bcoz of its Java-like API.
But there is one more XML data binding framework known as "Castor" which can handle all the marshalling and unmarshalling of object to XML for you automatically or with the help of a mapping file. Castor is indeed a cool XML data binding framework.

Furthur reading:
OnJava tutorial

http://www.castor.org/

This post originally appeared on an external website

by narendrn

When we go the default GAC folder in Windows located at : "C:\WINNT\assembly" we see the dlls that are registered in the GAC. But what if we have different versions of the same DLL registered in the GAC? How can a single Windows folder allow 2 files with the same name to exist?

To understand this, go to the DOS prompt and navigate to "C:\WINNT\assembly". Do a 'dir' command and see the contents. There would be a 'GAC' folder and inside the folder, there is a folder for each Assembly. Each assembly folder has a folder for each version. Inside this version folder there is the actual DLL. Try this out :)

This post originally appeared on an external website

by lukcad

For everyone application Microsoft.Net creates the temporary directory in section of the corresponding application. For example, you can find it for Framework version of v1.1.4332 by this way: C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\Temporary ASP.NET Files

There you will find all your server applications running on your server, also you will find that in every directory of your running application is saved copy of assembly. In assembly we will find the copy of libraries from bin directory of virtual catalog of current application.

So you can image how a lot of structured directories also will created till your application is runnig. So if you have a lot of virtual catalogues then not wonder why you need to have mirror copy of your hard disk.;-) Some programmers love to make image disks for repair system, but i afraid with Microsoft.NET it is trick is not suitable.

Sincerely, LukCAD

by narendrn

I often wondered what was the advantage of compiling files into .net modules and linking them together using the Al.exe tool. I also noticed that al.exe actually only creates a stub dll, the netmodule files really have to be physically deployed too.

I guess the advantage of compiling to netmodules are:

  • Each module may be independently developed by a separate set of developers.
  • Each module may be written in any .net language.
  • A multimodule assembly does have memory advantages: if you put rarely-usedtypes in one module, that module will only be loaded when one of its types is referenced, when the type is first used. If the rarely-used types are never used in an execution of the assembly, the module is not loaded.

This post originally appeared on an external website

by narendrn

After authenticating a user using forms authentication, we may want to restict access to certain parts of the website to certain users - i.e. Authorize users.

To implement Role based authorization we would need to set up a database containing info about which role a user belongs to. Then we need to construct a Principal object specifying which role the user belongs to and assign it to the HttpContext user property.

An excellent article discussing this concept is at:
http://aspnet.4guysfromrolla.com/articles/082703-1.aspx

This post originally appeared on an external website

by narendrn

I often used to write some utility classes that would simplify the coding of most used tasks.
For e.g. accessing a DataReader for binding to a grid, we have to write all the database plumbing code again and again.
Hence MS has released some utility wrapper classes that can be used to simplify such routine tasks. A good article explaining this is at http://aspnet.4guysfromrolla.com/articles/070203-1.aspx

The application blocks can be downloaded from http://www.microsoft.com/downloads

This post originally appeared on an external website

by narendrn

I have been interested in datawarehousing concepts for a long time, but unfortunately never got a chance to work on them. Here are some common concepts U need to know to understand any datawarehousing glibber.

Data Warehousing: An enterprise-wide implementation that replicates data from the same publication table on different servers/platforms to a single subscription table. This implementation effectively consolidates data from multiple sources.

Data Warehouse: A data warehouse is a subject oriented, integrated, non volatile, time variant collection of data. The data warehouse contains atomic level data and summarized data specifically structured for querying and reporting.

Data Mining: The process of finding hidden patterns and relationships in data. For instance, a consumer goods company may track 200 variables about each consumer. There are scores of possible relationships among the 200 variables. Data mining tools will identify the significant relationships.

OLAP (On-Line Analytical Processing)

Describes the systems used not for application delivery, but for analyzing the business, e.g., sales forecasting, market trends analysis, etc. These systems are also more conducive to heuristic reporting and often involves multidimensional data analysis capabilities.

OLTP (OnLine Transaction Processing)

Describes the activities and systems associated with a company's day-to-day operational processing and data (order entry, invoicing, general ledger, etc.).

Data Dictionary:
a software tool for recording the definition of data, the relationship of one category of data to another, the attributes and keys of groups of data, and so forth.

This post originally appeared on an external website

by narendrn

I still see a lot of applications that are vulnerable to SQL injection attacks bcoz of usage of dynamic SQL using string concatenation. It is a common myth that to circumvent SQL injection attacks we 'have' to use stored procedures.
Even if we are using dynamic SQL, it is pretty simple to avoid these attacks.

In .NET the following techniques can be used:
Step 1. Constrain input - Validate input using client side validation and server side validation (e.g. using regular expressions)
Step 2. Use parameters with stored procedures. There is one caveat with Stored procedures. If Ur stored procedure uses the 'EXEC' command which takes a string, then the same vulnerability exists there too.
Step 3. Use parameters with dynamic SQL. (Yepppiee...in .NET it's so simple to have named parameters even for dynamic SQL)

Code snippet :
-----------------------------------
SqlDataAdapter myDataAdapter = new SqlDataAdapter( "SELECT au_lname, au_fname FROM Authors WHERE au_id = @au_id", connection);

myCommand.SelectCommand.Parameters.Add("@au_id", SqlDbType.VarChar, 11); myCommand.SelectCommand.Parameters["@au_id"].Value = SSN.Text;

-----------------------------------

Other important points to be considred are using a least-privileged database account and avoiding disclosing error information to the user.
A good article discussing this is at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/paght000002.asp

In Java, one can use 'Prepared Statements' and 'Stored Procedures' to prevent SQL injection attacks.

This post originally appeared on an external website

by narendrn

During i8n and localization, we often come across basic fundamental issues such as:
- How many bytes make a character?
- How many characters/bytes are present in a string?

Each character gets encoded into bytes according to a specific charset. For e.g. ASCII uses 7 bit encoding, i.e. each char is represented by 7 bits. ANSI/Cp1521 uses 8-bit encoding, Unicode uses 16 bit encoding. UTF-8, which is a popular encoding set on the internet is a multibyte Unicode charset. So if someone asks - how many bytes make a character - the answer is - it depends on the charset used to encode the character.

Another interesting point in Java is the difference btw a 'char' and a character.
When we do "String.length()" in Java, we get the number of chars in the string. But a Unicode character may be made up of more than one 'char'.
This blog throws light on this concept: http://forum.java.sun.com/thread.jspa?threadID=671720

Snippet from the above blog:
---------------------------------
A char is not necessarily a complete character. Why? Supplementary characters exist in the Unicode charset. These are characters that have code points above the base set, and they have values greater than 0xFFFF. They extend all the way up to 0x10FFFF. That's a lot of characters. In Java, these supplementary characters are represented as surrogate pairs, pairs of char units that fall in a specific range. The leading or high surrogate value is in the 0xD800 through 0xDBFF range. The trailing or low surrogate value is in the 0xDC00 through 0xDFFF range. What kinds of characters are supplementary? You can find out more from the Unicode site itself.

So, if length won't tell me home many characters are in a String, what will? Fortunately, the J2SE 5.0 API has a new String method: codePointCount(int beginIndex, int endIndex). This method will tell you how many Unicode code points are between the two indices. The index values refer to code unit or char locations, so endIndex - beginIndex for the entire String is equivalent to the String's length. Anyway, here's how you might use the method:

int charLen = myString.length();
int characterLen = myString.codePointCount(0, charLen);

This post originally appeared on an external website

by narendrn

I still see many people with a lot of myths about Unicode. I guess the reason for this is that a lot of people still feel that Unicode is a encoding format that uses 16 bits to represent a character.
Let's put a few things in perspective here:

Unicode is a standard which has defined a character code for every character in most of the speaking languages in the world. Also it has defined a character code for items such as scientific, mathematical, and technical symbols, and even musical notation. These character codes are also known as code points.

Unicode characters may be encoded at any code point from U+0000 to U+10FFFF, i.e. Unicode reserves 1,114,112 (= 220 + 216) code points, and currently assigns characters to more than 96,000 of those code points. The first 256 codes precisely match those of ISO 8859-1, the most popular 8-bit character encoding in the "Western world"; as a result, the first 128 characters are also identical to ASCII.

The number of bits used to represent each code point may differ - e.g. 8 bits, 16 bits.
The size of the code unit used for expressing those code points may be 8 bits (for UTF-8), 16 bits (for UTF-16), or 32 bits (for UTF-32).
So what this means is that there are several formats for storing Unicode code points. When combined with the byte order of the hardware (BE or LE), they are known officially as "character encoding schemes." They are also known by their UTF acronyms, which stand for "Unicode Transformation Format"

UTF-8 is widely used because the first 128 bits in the byte are ASCII, and although up to four bytes can be used, only one byte is required for use in the English speaking world. UTF-16 and UTF-32 use a fixed number of bytes.

So to put in other words, Unicode text can be represented in more than one way, including UTF-8, UTF-16 and UTF-32. So, hey...what's this UTF ?

A Unicode transformation format (UTF) is an algorithmic mapping from every Unicode code point to a unique byte sequence. UTF-8 is most common on the web. UTF-16 is used by Java and Windows. UTF-32 is used by various Unix systems. The conversions between all of them are algorithmically based, fast and lossless. This makes it easy to support data input or output in multiple formats, while using a particular UTF for internal storage or processing.

For more information visit http://www.unicode.org/faq/

This post originally appeared on an external website

by narendrn

There is a new class in Java5.0 - StringBuilder class that can be used whenever we are doing some heavy-duty string manipulation and we do not wish to get bogged down by the immutable property of strings.
But then what happened to the old good StringBuffer class. Well, it's still there.
Here's what the JavaDoc says:

This StringBuilder class provides an API compatible with StringBuffer, but with no guarantee of synchronization. This class is designed for use as a drop-in replacement for StringBuffer in places where the string buffer was being used by a single thread (as is generally the case). Where possible, it is recommended that this class be used in preference to StringBuffer as it will be faster under most implementations.

Instances of StringBuilder are not safe for use by multiple threads. If such synchronization is required then it is recommended that StringBuffer be used.

This post originally appeared on an external website

by narendrn

I have often used thread pools in my applications. My favourite opensource thread pool was the PooledExecutor class written by Doug Lea (available here )

But it's also important to understand when to use thread pools and when not to use. I recently came across a good article on the web - http://www-128.ibm.com/developerworks/java/library/j-jtp0730.html

In this article the author argues that using thread pools makes sense whenever U have a large number of tasks that need to be processed and each task is short-lived. For e.g. Webservers, FTP servers etc.
But when the tasks are long running and few in number it may make sense to actually spawn a thread for each task. Another common threading model is to have a single background thread and task queue for tasks of a certain type. AWT and Swing use this model, in which there is a GUI event thread, and all work that causes changes in the user interface must execute in that thread.

Java 5.0 has come up with a new package java.util.concurrent that contains a lot of cool classes if U need to implement threading in Ur applications.

This post originally appeared on an external website

by narendrn

Quite often, we may have to use object pooling to conserve resources and increase performance.
I came across some pretty good abstract components that can be used to develop a object pooling mechanism.

Check out the followling link to see all the classes that are available for use:
http://jakarta.apache.org/commons/pool/

An interesting aspect of the package was the clear separation of object creation from the way the objects are pooled. There is a PoolableObjectFactory interface that provides a generic interface for managing the lifecycle of a pooled instance.

By contract, when an ObjectPool delegates to a PoolableObjectFactory : -

makeObject is called whenever a new instance is needed.
activateObject is invoked on every instance before it is returned from the pool.
passivateObject is invoked on every instance when it is returned to the pool.
destroyObject is invoked on every instance when it is being "dropped" from the pool (whether due to the response from validateObject, or for reasons specific to the pool implementation.)
validateObject is invoked in an implementation-specific fashion to determine if an instance is still valid to be returned by the pool. It will only be invoked on an "activated" instance.

This post originally appeared on an external website

by rstrydom

Hi,

I have found the performance in the jakarta commons pool to be very
poor, to such an extent that I have thrown it away for a custom written
solution.

Regards,

Rudi

by mjremijan

I would have to agree the performance is very bad.  Since I deal with building Strings a lot I thought it might be a good idea to create a pool of StringBuffers instead of creating new objects every time.  The performance with jakarta commons pool was terrible.  I don't know of it's just a matter of synchronization making it slow or the way the jakarta commons pool is designed.

by absolutezero

I have seen very specific uses for this object.  To increase performance, I will create a pool of managed connections to a datastore and recycle those.  This enhances performance as I don't have to deal with object creation, looking up connection parameters, authentication, and so forth... 

Performance issues some people have seen with the object pool might actually result from other factors.  Consider the types of objects a developer is pooling.  Does it make good engineering sense to place those objects in a pool?  Also examine how the pooled objects are constructed and operate as they may be causing the reduction in performance.

As for me, I have not experienced any problems using the apache objects.

by shahinianrazmik

Insert comment text here

It is a very bad idea (it is even problematic) to define methods as throwing generic Exception.

One must define some checked exceptions and use it instead of declaring the methods with throws Exception.

I hope that you do not need more details or examples on pitfalls of your declarations.

razmik@biodiscovery.com

by jslarochelle

I am surprised to see this discussion about object pooling since I have
read that this is a bad idea with latest VM and could actually worsen
performance by throwing off the GC (except for Thread object were
overhead of creation is actually quite large).

by narendrn

In VS.NET, whenever we add a web-reference to a webservice, what happens behind the scenes is that a proxy is created which contains the URL to the webservice.

Now if we need to shift the webservice to a production server, do we have to create the proxy again?. (The IP address of the webservice server would have changed)

Fortunately there is a easy way out. Just right-click on the proxy in VS.NET and change its URL property from 'static' to 'dynamic'. And Voila !!!, VS.NET automatically adds code to the proxy class and web.config. The newly added code in the proxy will first check for a URL key in the web.config and use it to bind to the webservice.
So when we move the webservice to a different server, we just have to change the URL in the web.config file.

This post originally appeared on an external website

by narendrn

Most of us have worked on transactions one time or the other. Transactions can be broadly classified as local or distributed.
Local transactions are of the simplest form in which the application peforms CRUD operations with a single datasource/database. The simplest form of relational database access involves only the application, a resource manager, and a resource adapter. The resource manager can be a relational database management system (RDBMS), such as Oracle or SQL Server. All of the actual database management is handled by this component.
The resource adapter is the component that is the communications channel, or request translator, between the "outside world," in this case the application, and the resource manager. In Java applications, this is a JDBC driver.

In a distributed transaction, the transaction accesses and updates data on two or more networked resources, and therefore must be coordinated among those resources.
These resources could consist of several different RDBMSs housed on a single sever, for example, Oracle, SQL Server, and Sybase; or they could include several instances of a single type of database residing on a number of different servers. In any case, a distributed transaction involves coordination among the various resource managers. This coordination is the function of the transaction manager. The transaction manager is responsible for making the final decision either to commit or rollback any distributed transaction. A commit decision should lead to a successful transaction; rollback leaves the data in the database unaltered

The first step of the distributed transaction process is for the application to send a request for the transaction to the transaction manager. Although the final commit/rollback decision treats the transaction as a single logical unit, there can be many transaction branches involved. A transaction branch is associated with a request to each resource manager involved in the distributed transaction. Requests to three different RDBMSs, therefore, require three transaction branches. Each transaction branch must be committed or rolled back by the local resource manager. The transaction manager controls the boundaries of the transaction and is responsible for the final decision as to whether or not the total transaction should commit or rollback. This decision is made in two phases, called the Two-Phase Commit Protocol.

In the first phase, the transaction manager polls all of the resource managers (RDBMSs) involved in the distributed transaction to see if each one is ready to commit. If a resource manager cannot commit, it responds negatively and rolls back its particular part of the transaction so that data is not altered.

In the second phase, the transaction manager determines if any of the resource managers have responded negatively, and, if so, rolls back the whole transaction. If there are no negative responses, the translation manager commits the whole transaction, and returns the results to the application.

This post originally appeared on an external website

by narendrn

Recently, Microsoft's Pattern & Practices Group has released the Enterprise Library, a large configurable and extensible software library that consists of seven integrated application blocks.

For more information and download visit http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/entlib.asp

The application blocks that comprise the Enterprise Library are the following:
Caching Application Block. This application block allows developers to incorporate a local cache in their applications.
Configuration Application Block. This application block allows applications to read and write configuration information.
Data Access Application Block. This application block allows developers to incorporate standard database functionality in their applications.
Cryptography Application Block. This application block allows developers to include encryption and hashing functionality in their applications.
Exception Handling Application Block. This application block allows developers and policy makers to create a consistent strategy for processing exceptions that occur throughout the architectural layers of enterprise applications.
Logging and Instrumentation Application Block. This application block allows developers to incorporate standard logging and instrumentation functionality in their applications.
Security Application Block. This application block allows developers to incorporate security functionality in their applications. Applications can use the application block in a variety of situations, such as authenticating and authorizing users against a database, retrieving role and profile information, and caching user profile information.

This post originally appeared on an external website

by narendrn

Most databases have built-in functions, such as GetDate, DateAdd, and ObjectName. Such built-in functions are useful, but you can?t alter their functionality in any way, and that?s why UDFs are so powerful and necessary. UDFs allow you to add custom solutions for unique application-specific problems.
A UDF is actually a kind of subroutine that contains T-SQL statements and can return a scalar value or a table value. Hence U can call a UDF from a SELECT statement. Whereas a Stored Procedure needs to be invoked using the 'EXEC' command.
So when to use what? The answer depends on the problem situation. If you have an operation such as a query with a FROM clause that requires a rowset be drawn from a table or set of tables, then a function will be your appropriate choice. However, when you want to use that same rowset in your application the better choice would be a stored procedure.

The other differences btw a UDF and a Stored Procedure are as follows:

  • A stored prodecure supports output parameters, whereas a UDF does not.
  • A UDF can have a return value (scalar or table). A SPROC can create a table, but cannot return it. It can only return a integer (as status code)
  • SPROCs are powerful enough to change server environment variables. UDFs are not.
  • SPROCs can be used in the FOR XML clause but UDFs can't be.
  • Whenever an error occurs in a UDF, the function stops immediately. In a SPROC, we must include error handling code, or the next statement is processed.
  • U can 'join' to a UDF, but cannot 'join' to a SPROC.

This post originally appeared on an external website

by narendrn

Came across a wonderful article on MSDN which contrasts the use of DataReader Vs DataSet.
The article is available at : http://msdn.microsoft.com/msdnmag/issues/04/06/DataPoints/

Salient points of the article:

  • A DataReader is a stream of data that is returned from a database query. When the query is executed, the first row is returned to the DataReader via the stream. The stream then remains connected to the database, poised to retrieve the next record. The DataReader reads one row at a time from the database and can only move forward, one record at a time. As the DataReader reads the rows from the database, the values of the columns in each row can be read and evaluated, but they cannot be edited.
  • The DataReader can only get its data from a data source through a managed provider. The DataSet can also get its data from a data source via a managed provider, but the data source can also be loaded manually, even from an XML file on a hard drive.
  • The DataReader supports access to multiple resultsets, one at a time, in the order they are retrieved. (Methods to checkout are NextResult() and Read() )
  • When the SqlDataAdapter's Fill method is executed it opens its associated SqlConnection object (if not already open) and issues its associated SqlCommand object against the SqlConnection. Behind the scenes, a SqlDataReader is created implicitly and the rowset is retrieved one row at a time in succession and sent to the DataSet. Once all of the data is in the DataSet, the implicit SqlDataReader is destroyed and the SqlConnection is closed.
  • The DataSet's disconnected nature allows it to be transformed into XML and sent over the wire via HTTP if appropriate. This makes it ideal as the return vehicle from business-tier objects and Web services. A DataReader cannot be serialized and thus cannot be passed between physical-tier boundaries where only string (XML) data can go.
  • There are other times when a DataReader can be the right choice, such as when populating a list or retrieving 10,000 records for a business rule. When a huge amount of data must be retrieved to a business process, even on a middle tier, it can take a while to load a DataSet, pass the data to it on the business tier from the database, and then store it in memory. The footprint could be quite large and with numerous instances of it running (such as in a Web application where hundreds of users may be connected), scalability would become a problem. If this data is intended to be retrieved and then traversed for business rule processing, the DataReader could speed up the process as it retrieves one row at a time and does not require the memory resources that the DataSet requires.

This post originally appeared on an external website

by narendrn

A few years back, I often used to lament on the lack of packet sniffing API's available in Java or C#.NET. I knew of the WinPcap library, but I wanted a wrapper around it. Fortunately some good guys in the open source community have build these wrappers around WinPcap.

Check out the following links:
http://www.tamirgal.com/home/dev.aspx?Item=SharpPcap (A C# API)
http://sourceforge.net/projects/jpcap (A Java API)
http://www.winpcap.org/

http://www.tcpdump.org/

This post originally appeared on an external website

by narendrn

Recently, I wanted to see the hex dump of a binary file on a server. The server machine did not have my favourite hex GUI editors. I struggled to download a hex dump utility from the web, when my friend showed me how to use the 'debug.exe' tool in DOS to get a hex dump of a file.

On the DOS prompt type :
C:\>debug
-d
-----------------------
The 'd' (dump) option would dump the first 128 bytes. Typing 'd' again would give the hex dump of the next block of bytes.

U can type '?' to get a list of all the commands. Type 'q' to quit from debug mode.

This post originally appeared on an external website

by narendrn

Cursors are database objects that allow us to manipulate data in a set in a row-by-row basis or on a group of rows at one time. Cursors are quite popular among database developers as the row can be updated at the same time itself. There is no need to fire a separate SQL query.

But there is also a lot of confusion regarding the exact definition of cursors, because different people use it in different contexts. For e.g. when a database administrator talks of cursors, he means the server-side cursor he uses inside stored procedures. But if a JDBC application developer talks about a cursor, he may mean the pointer in the JDBC ResultSet object.
I did a bit of reseach on the web, and finally cleared a few cobwebs from the head. Here's a snapshot of what I learned:

  • There are implicit cursors and explicit cursors. An implicit cursor is - well, that's just a piece of memory used by the database server to work with resulsets internally. Implicit cursors are not accessible via an externally exposed API, whereas explicit cursors are.
  • Server-side cursors and Client-side cursors: The difference between a client-side cursor and a server-side cursor in classic ADO is substantial and can be confusing. A server-side cursor allows you to manipulate rows on the server through calls on the client, usually storing all or a portion of the data in TEMPDB. The client-side cursor fetches data into a COM object on the client. Its name comes from the fact that the buffered data on the client exhibits cursor-like behaviors you can scroll through and, potentially, update it. The behavior difference manifests itself in a few ways. Fetching a large resultset into a client cursor causes a big performance hit on the initial fetch, and server cursors result in increased memory requirements for SQL Server and require a dedicated connection all the time you're fetching. Client-side cursors can be dangerous because using them has the side effect of retrieving all records from the server that are a result of your command/SQL statement. If you execute a procedure or SQL statement that could retrieve 10,000 records, and use a client-side cursor, you had better have enough memory to hold 10,000 records on the client. Also, control will not return to your code/application until all of these records are retrieved from the server, which can make your application appear slow to users. If you suspect that you are going to retrieve a large number of rows, client-side cursors are not the way to go.
  • DYNAMIC, STATIC, and KEYSET cursors: Dynamic cursors will show changes made on the base table as you scroll through the cursor. Static cursors copy the base table, to the tempdb. The cursor then reads from the tempdb, so any changes happening on the base table will not be reflected in the cursors scrolling. Keysets are in between Dynamic and Static cursors. A keyset will copy the base table's selected records keys into the tempdb, so the cursor will select the rows from the tempdb, but the data from the base table. So change to the base table will be seen, but new record inserts will not be.

Next, I tried to find out how these cursors can be manipulated using .NET and Java APIs.

In .NET, the DataReader object acts as a wrapper class for a server-side cursor. The DataReader provides a read-only and forward-only cursor. More info about this is here.

In JDBC, we handle everything using ResultSet objects. Since JDBC 2.0/3.0, we have ResultSets that are scrollable backwards and also that allow update operations. So I believe if U configure a ResultSet to be scrollable backwards and allow update opeartions, U are essentially dealing with a server-side cursor.

Code snippet:

Statement stmt = con.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_READ_ONLY);

ResultSet srs = stmt.executeQuery("SELECT COF_NAME, PRICE FROM COFFEES");

---------------------------------------------------------------------------------

The above code makes the ResultSet as scroll-sensitive, so it will reflect changes made to it while it is open. This is the equivalent of a dynamic cursor. In a dynamic cursor, the engine repeats the query each time the cursor is accessed so new members are added and existing members are removed as the database processes changes to the database.

The second argument is one of two ResultSet constants for specifying whether a result set is read-only or updatable: CONCUR_READ_ONLY and CONCUR_UPDATABLE.
Specifying the constant TYPE_FORWARD_ONLY creates a nonscrollable result set, that is, one in which the cursor moves only forward. If you do not specify any constants for the type and updatability of a ResultSet object, you will automatically get one that is TYPE_FORWARD_ONLY and CONCUR_READ_ONLY (as is the case when you are using only the JDBC 1.0 API). You might keep in mind, though, the fact that no matter what type of result set you specify, you are always limited by what your DBMS and driver actually provide.

This post originally appeared on an external website

by narendrn

A multidimensional database (MDB) is a type of database that is optimized for data warehouse and online analytical processing (OLAP) applications. Multidimensional databases are frequently created using input from existing relational databases.

A multidimensional database - or a multidimensional database management system (MDDBMS) - implies the ability to rapidly process the data in the database so that answers can be generated quickly. A number of vendors provide products that use multidimensional databases. Approaches to how data is stored and the user interface vary.

Conceptually, a multidimensional database uses the idea of a data cube to represent the dimensions of data available to a user. For example, "sales" could be viewed in the dimensions of product model, geography, time, or some additional dimension. In this case, "sales" is known as the measure attribute of the data cube and the other dimensions are seen as feature attributes. Additionally, a database creator can define hierarchies and levels within a dimension (for example, state and city levels within a regional hierarchy).

More information can be at:
http://flame.cs.dal.ca/~panda/overview.html

http://www.dba-oracle.com/cou_mdb_class.htm

This post originally appeared on an external website

by awolfe_ii

I never quite got the idea of MDBs.  It always struck me that each table column is a dimension anyway.

This post supports my suspicion that MDB is primarily a query/UI mechanism rather than a form of new data modeling construct, perhaps supported by special SQL row/group functions.  Granted, of course, that special DB features under the covers can greatly accelerate such highly specialized types of queries, but as a data-model-oriented guy, that doesn't sound like 'a new type of database.'

So am I understanding MDBs correctly?

by narendrn

While working on projects, often I had tussles with erst-while database developers who used to drive the design of the system using Data Models. Most of them came from non-OO background and were apt in ER diagrams. But the porblem is that a Data Model cannot and should not drive the OO model. I came across this excellent article by Scott Ambler discussing this dilemma.

http://www.agiledata.org/essays/drivingForces.html

In the above article, the author argues that object schemas can be quite different from physical data schemas. Using two lucid examples he shows how different object schemas can map to the same data schema and how it is possible for a single object schema to correctly map to several data schemas.
There are situations where OO varies significantly from ERD, for example in cases of inheritance and non-persistent classes.
Aslo, OO models can convey more specification on the class diagram in representing certain things like associations than ERD can.

I personally believe that if U are building a OO system, then we should first go for OO Class modelling and then think about resolving the OO-relational database impedance mismatch.

This post originally appeared on an external website

by narendrn

I decided to list down all the available tools for different phases of a SDLC. Here it goes:

Requirements Management:

- DOORS: Dynamic Object Oriented Requirements System
- RequisitePro : A requirements management tool from Rational Software
- Requirements Traceability Management (RTM)

Design:

- ArgoUML: http://argouml.tigris.org/

-Rational Rose http://www.rationalrose.com/

-Enterprise Architect http://www.sparxsystems.com.au/

-Rhapsody http://www.ilogix.com/homepage.aspx

-ERwin http://www3.ca.com/solutions/Product.aspx?ID=260

-Rational Rose Real Time http://www-128.ibm.com/developerworks/rational/products/rose

-Describe http://www.embarcadero.com/products/describe/

-MagicDraw http://www.magicdraw.com/

-SmartDraw: http://www.smartdraw.com/examples/software-erd/

-Visio : http://www.mvps.org/visio/

Code Generation:

-OptimalJ: http://www.compuware.com/products/optimalj/

- Rational Application Developer

-Object Frontier

-JVider is Java Visual Interface Designer which helps with GUI building for Swing applications

Unit Testing:

-Jtest: http://www.parasoft.com/jsp/home.jsp

-JUnit: http://www.junit.org/

Load Testing:

SilkPerformer is load performance testing tool.
LoadRunner

JMeter

ANTS Load

Microsoft Web Application Stress Tool

Web Testing/ GUI testing:

SilkTest from Segue Software
Rational ROBOT

WinRunner

Configuration Management Tools:

Concurrent Version System - CVS

Subversion

Clearcase

Visual SourceSafe

Documentation:

Framemaker

Will keep augmenting this list...

This post originally appeared on an external website

by narendrn

This seems to a favourite question during interviews - How to delete duplicate rows in a table?
Well, we can have many strategies:

- Capture one instance of the unique rows using a SELECT DISTINCT .., and dump the results into a temp table. Delete all of the rows from the original table and then insert the rows from the temp table back into the original table.

- If there is an identity column, the we can also write a query to get all the rows that have duplicate entries :
SELECT Field1, Field2, Count(ID)
FROM Foo1
GROUP BY Foo1.Field1, Foo1.Field2
HAVING Count(Foo1.ID) > 1
-----------------------------
Then loop thru the resultset and get a cursor for the query
"SELECT Field1, Field2, ID
FROM Foo1
WHERE Field1 = @FIELD1 and Field2 = @FIELD2".
Use the cursor to delete all the duplicate rows returned except one.

The following links discuss the above techniques and some more:

sql-server-performance

databasejournal

aspfaqs

This post originally appeared on an external website

by narendrn

Databases support different types of locking strategies. We have optimistic locking strategies where concurrency would be more, but data integrity would be less. We have pessimistic locking strategies where concurrency is less, but there is more data integrity. Basically when it comes to locking and transaction isolation levels, its a trade-off between data integrity and concurrency.

Databases supports various levels of lock granularity. The lowest level is a single row. Sometimes, the database doesn't have enough resources to lock each individual
row. In such cases, the database can acquire locks on a single data or index page, group of pages, or an entire table. The granularity of locks depends on the memory available to the database. Servers with more memory can support more concurrent users because they can acquire and release more locks.

We can give locking hints to the database while sending queries that would help us override locking decisions made by the database. For instance, in SQL Server. we can specify the ROWLOCK hint with the UPDATE statement to convince SQL Server to lock each row affected by that data modification.

While dealing with transactions, the type of locking used depends on the Transaction Isolation Level specified for that particular transaction.
SQL Server supports implicit and explicit transactions. By default, each INSERT, UPDATE, and DELETE statement runs within an implicit transaction.Explicit transactions, on the other hand, must be specified by the programmer. Such transactions are included in
BEGIN TRANSACTION ... COMMIT TRANSACTION block

There are two common types of locks present in all databases- Shared locks and Exclusive locks.
Shared locks (S) allow transactions to read data with SELECT statements. Other connections are allowed to read the data at the same time; however, no transactions are allowed to modify data until the shared locks are released.
Exclusive locks (X) completely lock the resource from any type of access including reads. They are issued when data is being modified through INSERT, UPDATE and DELETE statements

Locking/Transaction Isolation levels solve the following concurrency problems:

  • Lost Updates: Lost updates occur when two or more transactions select the same row and then update the row based on the value originally selected. Each transaction is unaware of other transactions. The last update overwrites updates made by the other transactions, which results in lost data.
  • Dirty Read: Uncommitted dependency occurs when a second transaction selects a row that is being updated by another transaction. The second transaction is reading data that has not been committed yet and may be changed by the transaction updating the row.
  • Nonrepeatable Read: Inconsistent analysis occurs when a second transaction accesses the same row several times and reads different data each time.
  • Phantom reads: These occur when an insert or delete action is performed against a row that belongs to a range of rows being read by a transaction. The transaction's first read of the range of rows shows a row that no longer exists in the second or succeeding read, as a result of a deletion by a different transaction. Similarly, as the result of an insert by a different transaction, the transaction's second or succeeding read shows a row that did not exist in the original read.

These concurrency problems can be solved using the correct transaction isolation level in our programs. To balance between high concurrency and data integrity, we can choose from the following isolation levels:

  • READ UNCOMMITTED: This is the lowest level where transactions are isolated only enough to ensure that physically corrupt data is not read. Does not protect against dirty read, phantom read or non-repeatable reads.
  • READ COMMITTED: The shared lock is held for the duration of the transaction, meaning that no other transactions can change the data at the same time. Protects against dirty reads, but the other problems remain.
  • REPEATABLE READ: This setting disallows dirty and non-repeatable reads. But phantom reads are still possible.
  • SERIALIZABLE: This is the highest level, where transactions are completely isolated from one another. Protects against dirty read, phantom read and non-repeatable read.

There are other types of locks available in the database such as 'update lock', 'intent lock', 'schema lock' etc. The update lock prevents deadlocks in the database. For more information about these check out the following links:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/acdata/ac_8_con_7a_6fhj.asp

http://www.awprofessional.com/articles/article.asp?p=26890&rl=1

This post originally appeared on an external website

by narendrn

To access an internet site thru a .NET program U may have to go thru Ur company proxy.
Here's the sample code for doing so:

string query="

http://narencoolgeek.blogspot.com

";

WebProxy proxyObj = new WebProxy("10.10.111.19", 8080) ;

NetworkCredential networkCredential = new NetworkCredential("username", "secret_password") ;

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(query) ; proxyObj.Credentials = networkCredential ;

req.Proxy = proxyObj ;

For proxy setting in Java programs click here.

This post originally appeared on an external website

by narendrn

The Synchronizer Token pattern addresses the problem of duplicate form submissions. A synchronizer token is set in a user's session and included with each form returned to the client. When that form is submitted, the synchronizer token in the form is compared to the synchronizer token in the session. The tokens should match the first time the form is submitted. If the tokens do not match, then the form submission may be disallowed and an error returned to the user. Token mismatch may occur when the user submits a form, then clicks the Back button in the browser and attempts to resubmit the same form.

On the other hand, if the two token values match, then we are confident that the flow of control is exactly as expected. At this point, the token value in the session is modified to a new value and the form submission is accepted.

Struts has inbuild support for the Synchronizer Token pattern. Method such as "saveToken()" and "isTokenValid()" are present in the ActionForm class.

Other ways of preventing a duplicate submit is using javascript to disable the submit button after it has been pressed once. But this does not work well with all browsers.

This post originally appeared on an external website

by narendrn

Often I have heard people complain that Swing Apps are slow. I also have had some bad experiences with slow Swing Apps, but then I have also seen some very cool high performant applications. So, what's the secret of these fast Swing apps. I think the reason can be found in James Gosling's answer:

The big problem that Swing has is that it's big and complicated. And the reason that it's big and complicated is that it is the most fully featured, flexible, whatever-way-you-want-to-measure-it UI [user interface] tool kit on the planet. You can do the most amazing things with Swing. It's sort of the 747 cockpit of UI design tool kits, and flying a 747 is a little intimidating, and what people really want is a Cessna.

It is definately possible to write good speedy Swing applications, but U need to be a master of Swing and threading and know all the potholes which hog memory.

Here are a few links which contain some good info on fast Swing apps:

http://www.swingwiki.org/table_of_contents

http://www.javalobby.org/articles/swing_slow/index.jsp

http://www.clientjava.com/blog

http://www.javaworld.com/channel_content/jw-awt-index.shtml

http://www.javalobby.org/eps/galbraith-swing-1/index.html

This post originally appeared on an external website

by narendrn

I have had some experience with some O/R mapping tools such as TopLink and Hibernate.
But I did not know that the market is flooded with a variety of OR tools.
Here are a few of them:

Abra - http://abra.sourceforge.net
BasicWebLib - http://basicportal.sourceforge.net/
Castor - http://castor.exolab.org
Cayenne - http://objectstyle.org/cayenne/
DataBind - http://databind.sourceforge.net
DB VisualArchitect - http://www.visual-paradigm.com/dbva.php
EnterpriseObjectsFramework - http://www.apple.com/webobjects/
Expresso - http://www.jcorporate.com
FireStorm - http://www.codefutures.com/products/firestorm
Hibernate - http://www.hibernate.org
Hydrate - http://hydrate.sourceforge.net
iBATIS DB Layer - http://www.ibatis.com
Infobjects ObjectDRIVER- http://www.infobjects.com
InterSystems Cache - http://www.intersystems.com
Jakarta ObjectRelationalBridge - http://db.apache.org/ojb
Java Ultra-Lite Persistence (JULP) - http://julp.sourceforge.net
Jaxor - http://jaxor.sourceforge.net
JdoGenie - http://www.jdogenie.com
JDOMax - http://www.jdomax.com
JDX - http://www.softwaretree.com
KodoJdo - http://www.solarmetric.com/Software/Kodo_JDO/
LiDO - http://www.libelis.com
O/R Broker - http://orbroker.sourceforge.net/
Persistence EdgeXtend - http://www.persistence.com/products/edgextend/
PlanetJ's WOW = http://planetjavainc.com/wow.html
PowerMapJdo - http://www.powermapjdo.com
Signsoft intelliBO - http://www.intellibo.com
SimpleOrm - http://www.SimpleOrm.org
SpadeSoft XJDO - http://www.spadesoft.com
Sql2java - http://sql2java.sf.net
Sun Microsystem certified (SunTone?) - FrontierSuite? for J2EE & J2SE - http://www.objectfrontier.com
Sun Microsystem certified (SunTone?) - FrontierSuite? for JDO - http://www.objectfrontier.com
The ProductivityEnvironmentForJava (PE:J) - http://www.hywy.com/
TopLink - http://otn.oracle.com/products/ias/toplink/content.html
VBSF - http://www.objectmatter.com

Also the following articles give some some good introduction to O-R mapping tools and their comparisions:
http://c2.com/cgi-bin/wiki?ObjectRelationalToolComparison

http://www.agiledata.org/essays/mappingObjects.html

http://madgeek.com/Articles/ORMapping/EN/mapping.htm

This post originally appeared on an external website

by narendrn

One interesting point about ClassLoaders is that it is possible for the same class to be loaded separately by different class-loaders, resulting in multiple copies of the class object.
I came across a good synopsis on classloaders here : http://www.onjava.com/pub/a/onjava/2003/04/02/log4j_ejb.html

Here are some snippets from the above article:

Classloaders, as the name suggests, are responsible for loading classes within the JVM. Before your class can be executed or accessed, it must become available via a classloader. Given a class name, a classloader locates the class and loads it into the JVM. Classloaders are Java classes themselves. This brings the question: if classloaders are Java classes themselves, who or what loads them?
When you execute a Java program (i.e., by typing java at a command prompt), it executes and launches a native Java launcher. By native, I mean native to your current platform and environment. This native Java launcher contains a classloader called the bootstrap classloader. This bootstrap classloader is native to your environment and is not written in Java. The main function of the bootstrap classloader is to load the core Java classes.

The JVM implements two other classloaders by default. The bootstrap classloader loads the extension and application classloaders into memory. Both are written in Java. As mentioned before, the bootstrap classloader loads the core Java classes (for example, classes from the java.util package). The extension classloader loads classes that extend the core Java classes (e.g., classes from the javax packages, or the classes under the ext directory of your runtime). The application classloader loads the classes that make up your application.

bootstrap classloader <-- extension classloader <-- application classloader

All three default classloaders follow the delegation model. Before a child classloader tries to locate a class, it delegates that task to a parent. When your application requests a particular class, the application classloader delegates this request to the extension classloader, which in turn delegates it to the bootstrap classloader. If the class that you requested is a Java core class, the bootstrap classloader will make the class available to you. However, if it cannot find the class, the request returns to the extension classloader, and from there to the application classloader itself. The idea is that each classloader first looks up to its parent for a particular class. Only if the parent does not have the class does the child classloader try to look it up itself.

In application servers, each separately-deployed web application and EJB gets its own classloader (normally; this is certainly the case in WebLogic). This classloader is derived from the application classloader and is responsible for that particular EJB or web application. (Note that if an application is deployed as an EAR file--a combination of EJB and webapps--it gets one classloader, no more). This new classloader loads all classes that the webapp or EJB require that are not already part the Java core classes or the extension packages. It is also responsible for loading and unloading of classes, a feature missing from the default classloaders. This feature helps in hot deploy of applications.

When WebLogic starts up, it uses the Java-supplied application classloader to load the classes that make up its runtime. It then launches individual classloaders, derived from the Java application classloader, which load the classes for individual applications. The individual classloaders are invisible to the classloaders of the other applications; hence, classes loaded for one particular application will not be seen by another application.

What if you want to make a single class available to all applications? Load it in a top-level classloader. This could be in the classpath of WebLogic. When WebLogic starts, it will automatically load this class in memory using the Java-supplied application classloader. All sub-application classloaders get access to it. However, the negatives of this approach are clear too. First, you lose the capability of hot deploy for this particular class in all individual applications. Second, any change in this class means that the server needs to be restarted, as there is no mechanism for a Java application classloader to reload classes. You will need to weigh in the pros and cons before you take this approach.

WebLogic Server allows you to deploy newer versions of application modules such as EJBs while the server is running. This process is known as hot-deploy or hot-redeploy and is closely related to classloading.
Java classloaders do not have any standard mechanism to undeploy or unload a set of classes, nor can they load new versions of classes. In order to make updates to classes in a running virtual machine, the classloader that loaded the changed classes must be replaced with a new classloader. When a classloader is replaced, all classes that were loaded from that classloader (or any classloaders that are offspring of that classloader) must be reloaded. Any instances of these classes must be re-instantiated.

This post originally appeared on an external website

by narendrn

It took me sometime to appreciate the inheritance heirarchy in Log4J. It's quite simple to understand the Log Levels and their usage.

It is because of this heirarchy that we can have any arbitraty level of granularity in logging. For e.g. suppose we have 10 components in a distributed environment and we want to shutdown the logging of one component. This can be easily done in Log4J using config files. We can also disable logging at a class level - provided we have a Logger for that class - using the Logger.getLogger(this.class) method. Another example is if U want to switch off logging of one or more modules in a application.

We can even make dymanic changes in the configuration file at runtime and these changes would get reflected provided we have used the configureAndWatch() method.

This post originally appeared on an external website

by narendrn

While designing a systems, I often came across the moot point of choosing btw a heirarchical database and a relational flat table database. There are some domain problems, where heirarchical databases such as LDAP and MS Directory Services make sense over RDBs.

The hierarchical database model existed before the far more familiar relational model in early mainframe days. Hierarchical databases were blown away by relational versions because it was difficult to model a many-to-many relationship - the very basis of the hierarchical model is that each child element has only one parent element.

In his article, Scott Ambler makes the following statement:
Hierarchical databases fell out of favor with the advent of relational databases due to their lack of flexibility because it wouldn?t easily support data access outside the original design of the data
structure. For example, in the customer-order schema you could only access an order through a customer, you couldn?t easily find all the orders that included the sale of a widget because the schema isn?t designed to all that.

I found an article on the microsoft site which discusses the areas where directory services can be used in place of Relational databases.

Some good candidates for heirarchical databases are- White pages" information, User credential & security group information, Network configuration and service policies, Application deployment policies etc.

This post originally appeared on an external website

by sbailey

Nice post. I agree that network identity data (credentials, policies, location/reachability maps) are often well suited to hierarchical modeling due to clear requirements for distinction (namespaces) and containment (groups). It's often appropriate to separate modeling from implementation. Directories often conflate this two aspects inappropriately. I go into this in a little detail in a recent blog entry.

- Stu

by mleach

The mainframe hierarchical databases were actually just file systems with nested directories. Todays LDAP and object-oriented databases do in fact support many-to-many relationships, but require globally unique primary keys to do so.

The impedence mismatch of relational databases and object-oriented programming languages is revitalizing the concept of nested relational database (a hierarchical schema) to mirror todays development environments.

by jaqui

The "Historical" Hierarchical database is closer to themagnetic tape filesystem than to the current relational databases.
The relational databases and Structured query Language were developed
to make databases more usefull, once a random access media was
available. The original media used for data storage was the anciant
magnetic tape, where you could only work your way along the tape
checking each entry to see if it's the one you need.

The advent of hard drives, where you could access any record on the
drive by name, no need to scan all entries prior to it, inspired the
relational database model. This is actually a case of hardware
improving the productivity of software.

by dc guy

The distinction among the various database architectures is becoming moot as processing speeds approach Starburst. Datastores are physically laid out in patterns that facilitate maintenance, backup/recovery, integrity, currency, security, corporate turf, tradition, etc. Meanwhile the programs that read and write the data presume a relational pattern. It's the heroic middleware that tirelessly converts the relational operators into the appropriate physical commands and presents the results back to the user in the desired format.

Look at what you can do with OS/X Tiger. Type in a keyword and it will search the entire hard drive, paying no attention to architecture. If the keyword is embedded in a file or directory name, if it's part of the text of an e-mail or other document, if it's in the cell of a spreadsheet, or if it's in any other datastore in ASCII format, it will find it and tell you where it found it. And give you the entire list within a couple of seconds.

The next generation of processors will do the same thing on the scale of a corporate network in about the same time. And the nice folks who build Google and the other search engines want us to be able to do the same thing across the entire internet.

Relational, schmelational!

by narendrn

A friend of mine was asked in an interview to write a query to fetch the second highest marks in a table of students.

Well, I found out 2 ways to do this:

1) Using TOP to get the topmost 2 rows from the table in descending order and then again getting a TOP of the resultset.

SELECT TOP 1 *

FROM(SELECT TOP 2 *

FROM Table1

ORDER BY Marks desc)

ORDER BY Marks;

2) Using the MAX function. (I was suprised to see this work)

SELECT Max(Marks) FROM Table1

WHERE Marks Not In (SELECT MAX(Marks) FROM Table1);

This post originally appeared on an external website

by narendrn

I have often wanted to learn more about the class file format of Java class files. This study let me to interesting discoveries - regarding what debug information is stored in class files and how it is stored.

When we compile a Java source using the 'javac' exe, the generated class file by default contains some debug info - By default, only the line number and source file information is generated.
Hence when a stack trace is printed on the screen, we can see the source file name and the line number also printed on the screen. Also, when using log4J, I remember using a layout which can print the line number of each log statement - I bet log4J uses the debug infomation present in the class file to do this.

The javac exe also has a '-g' option - This option generates all debugging information, including local variables. To understand this better, I compiled a java source twice; once with the -g option and once without. I then decompiled both the class files to see how the decompiled output differs. The class file compiled with the debug option, showed all the local variable names same as in the original Java file, whereas the one without the debug option, had the local variables named by the decompiler as the local variable information was not stored in the class file.

Another interesting fact is that the class file can contain certain attributes that are non-standard; i.e. vendor specific. Suprised !!!...So was I..Please find below a snippet from the VM spec :


Compilers for Java source code are permitted to define and emit class files containing new attributes in the attributes tables of class file structures. Java Virtual Machine implementations are permitted to recognize and use new attributes found in the attributes tables of class file structures. However, all attributes not defined as part of this Java Virtual Machine specification must not affect the semantics of class or interface types. Java Virtual Machine implementations are required to silently ignore attributes they do not recognize.


For instance, defining a new attribute to support vendor-specific debugging is permitted. Because Java Virtual Machine implementations are required to ignore attributes they do not recognize, class files intended for that particular Java Virtual Machine implementation will be usable by other implementations even if those implementations cannot make use of the additional debugging information that the class files contain

This post originally appeared on an external website

by narendrn

Came across this cool article about Daemon Threads at http://www.absolutejava.com/main-articles/beware-the-daemons/

I knew that the VM does not exit as long as there is one non-daemon thread running. But there were a couple of new things I learned. Some snippets from the above article:

There are two ways a thread can become a daemon thread (or a user thread, for that matter) without putting your soul at risk. First, you can explicitly specify a thread to be a daemon thread by calling

setDaemon(true)

on a

Thread

object. Note that the setDaemon() method must be called before the thread's

start()

method is invoked.Once a thread has started executing (i.e., its start() method has been called) its daemon status cannot be changed.

The second technique for creating a daemon thread is based on an often overlooked feature of Java's threading behavior: If a thread creates a new thread and does not call setDaemon() to explicitlly set the new thread's "daemon status", the new thread inherits the "daemon-status" of the thread that created it. In other words, unless setDaemon(false) is called, all threads created by daemon threads will be daemon threads; similarly, unless setDaemon(true) is called, all threads created by user threads will be user threads.

This post originally appeared on an external website

by narendrn

Recently I came across the term "Screen Scraping" when interfacing with a legacy application. I was not really sure what it mean in the context of EAI.. A google search came up with a good explanation at http://en.wikipedia.org/wiki/Screen_scraping

As a concrete example of a classic screen scraper, consider a hypothetical legacy system dating from the 1960s -- the dawn of computerized data processing. Computer to user interfaces from that era were often simply text-based dumb terminals which were not much more than virtual teleprinters. (Such systems are still in use today, for various reasons.) The desire to interface such a system to more modern systems is common. An elegant solution will often require things no longer available, such as source code, system documentation, APIs, and/or programmers with experience in a 45 year old computer system. In such cases, the only feasible solution may be to write a screen scraper which "pretends" to be a user at a terminal. The screen scraper might connect to the legacy system via Telnet, emulator the keystrokes needed to navigate the old user interface, process the resulting display output, extract the desired data, and pass it on to the modern system.

This post originally appeared on an external website

by wayne m.

Actually, screen scraping is more common with more complex user interfaces based on things like IBM-style 3270 displays which did many things similar to HTML.  The issue is typically not a technology issue, but usually a funding and responsibility issue.

Screen scraping is usually chosen not because of lack of technical ability or knowledge to implement a more convenient interface, but because of political issues.  Screen scraping allows a group who wishes automated access to the data on another system to pull the data without requiring programming changes on the taret system.  In most projects, direct funding for interfaces is only provided to one of the systems involved.  The other system must essentially develop its side for free (out of its O&M budget).  Screen scraping shifts almost all of the develop cost to the user of the data with minimal consulting assistance from the provider.

by narendrn

In many design scenarios, asynchronous messaging is required- for integration purposes, for handling large volume of requests etc.
In the J2EE scenario, we often have to decide whether to write a simple JMS client or go for a MDB in a EJB container. Though, a simple JMS client is neat and easy to write, it has some disadvantages:
JMS has no built-in mechanism to handle more than one incoming request at a time. To support concurrent requests, you will need to extend the JMS client to spawn multiple threads, or launch multiple JVM instances, each running the application.On the downside, we have issues of security, transaction handling, and scalability.

The main advantage that MDBs have over simple JMS message consumers is that the EJB container can instantiate multiple MDB instances to handle multiple messages simultaneously.
A message-driven bean (MDB) acts as a JMS message listener. MDBs are different from session beans and entity beans because they have no remote, remote home, local, or local home interfaces. They are similar to other bean types in that they have a bean implementation, they are defined in the ejb-jar.xml file, and they can take advantage of EJB features such as transactions, security, and lifecycle management.

As a full-fledged JMS client, MDBs can both send and receive messages asynchronously via a MOM server. As an enterprise bean, MDBs are managed by the container and declaratively configured by an EJB deployment descriptor.
For instance, a JMS client could send a message to an MDB (which is constantly online awaiting incoming messages), which in turn could access a session bean or a handful of entity beans. In this way, MDBs can be used as a sort of an asynchronous wrapper, providing access to business processes that could previously be accessed only via a synchronous RMI/IIOP call.
Because they are specifically designed as message consumers and yet are still managed by the EJB container, MDBs offer a tremendous advantage in terms of scalability. Because message beans are stateless and managed by the container, they can both send and receive messages concurrently (the container simply grabs another bean out of the pool). This, combined with the inherent scalability of EJB application servers, produces a very robust and scalable enterprise messaging solution.

They cannot be invoked in any manner other than via a JMS message. This means that they are ideally suited as message consumers, but not necessarily as message producers. Message-driven beans can certainly send messages, but only after first being invoked by an incoming request. Also, MDBs are currently designed to map to only a single JMS destination. They listen for messages on that destination only.

When a message-driven bean's transaction attribute is set to Required, then the message delivery from the JMS destination to the message-driven bean is part of the subsequent transactional work undertaken by the bean. By having the message-driven bean be part of a transaction, you ensure that message delivery takes place. If the subsequent transactional work that the message-driven bean starts does not go to completion, then, when the container rolls back that transactional work it also puts the message back in its destination so that it can be picked up later by another message-driven bean instance.

This post originally appeared on an external website

by narendrn

Came across this wonderful explanation by Mike regarding XA transactions here

Excerpt:

An XA transaction, in the most general terms, is a "global transaction" that may span multiple resources. A non-XA transaction always involves just one resource. An XA transaction involves a coordinating transaction manager, with one or more databases (or other resources, like JMS) all involved in a single global transaction. Non-XA transactions have no transaction coordinator, and a single resource is doing all its transaction work itself (this is sometimes called local transactions).

XA transactions come from the X/Open group specification on distributed, global transactions. JTA includes the X/Open XA spec, in modified form. Most stuff in the world is non-XA - a Servlet or EJB or plain old JDBC in a Java application talking to a single database. XA gets involved when you want to work with multiple resources - 2 or more databases, a database and a JMS connection, all of those plus maybe a JCA resource - all in a single transaction. In this scenario, you'll have an app server like Websphere or Weblogic or JBoss acting as the Transaction Manager, and your various resources (Oracle, Sybase, IBM MQ JMS, SAP, whatever) acting as transaction resources. Your code can then update/delete/publish/whatever across the many resources. When you say "commit", the results are commited across all of the resources. When you say "rollback", _everything_ is rolled back across all resources.

The Transaction Manager coordinates all of this through a protocol called Two Phase Commit (2PC). This protocol also has to be supported by the individual resources. In terms of datasources, an XA datasource is a data source that can participate in an XA global transaction. A non-XA datasource generally can't participate in a global transaction (sort of - some people implement what's called a "last participant" optimization that can let you do this for exactly one non-XA item).

This post originally appeared on an external website

by narendrn

In one application scenario, it was required to process the message only once. The message contained a donation amount that needed to be put in the database. Using a Queue and AUTO_ACK semantics, we were able to achieve this. But there was a problem - What would happen if the JMS provider fails before the acknowledgement is received from client. It would send the message again when it comes up. How to avoid this ?

The consumer may receive the message again, because when delivery is guaranteed, it's better to risk delivering a message twice than to risk losing it entirely. A redelivered message will have the JMSRedelivered flag set. A client application can check this flag by calling the getJMSRedelivered() method on the Message object. Only the most recent message received is subject to this ambiguity.

This post originally appeared on an external website

by sjtech

I was looking for such software in December and happened to find
Wrapper on Sourceforge. I installed and tested on my laptop and it
works very nicely.  Easy to setup and run. Also nice for Tomcat
and their web site explains how to.

by narendrn

Quite often in messaging applications, we need to simulate a simple request-response paradigm using JMS. This can be achieved using temporary queues.

If you create a temporary destination and specify it as the value of the JMSReplyTo message header field when you send a message, the consumer of the message can use the value of the JMSReplyTo field as the destination to which it sends a reply and can also reference the original request by setting the JMSCorrelationID header field of the reply message to the value of the JMSMessageID header field of the request.

An important point is that the temporary queue is only valid till the connection object is open.

This post originally appeared on an external website

by narendrn

Web applications often need to have both http and https access; i.e. some URIs in the web application need to have HTTPS access and some can do with HTTP.

Hard-coding the URL address with https whereever required is not an elegant solution as this has to be reflected in all the links in all pages. An alternative soln will be to do a Response.redirect from the page with https in the URL. More detailed info is available at :
http://www.javaworld.com/javaworld/jw-02-2002/jw-0215-ssl.html

Also Struts has a extension through which we can specify which URLs need HTTPS access. For more info check out:
http://struts.apache.org/struts-taglib/ssl.html

http://sslext.sourceforge.net/

This post originally appeared on an external website

by narendrn

Recently while working on BEA Tuxedo server, I came across the term 'ATMI' - Application to Transaction Monitor Interface. I beleive Tux was quite a popular product for handling large number of transactions in a distributed environment - right from the early days when most business critical applications were written in Cobol.
ATMI is a procedural, non object oriented API that you use to write stateless services to manage transactions. This API is developed by the OpenGroup and forms the core service API of Tuxedo. The Application-Transaction Monitor Interface (ATMI) provides the interface between the COBOL application and the transaction processing system. This interface is known as ATMI and these pages specify its COBOL language binding. It provides routines to open and close resources, manage transactions, manage record types, and invoke request/response and conversational service calls.

But I believe today's TP monitors are based on Corba OTS (Object Transaction service) semantics. Corba OTS is used by any OO application as the underlying semantics for communications with resouce managers in a distributed transaction. Even JTS is the Java langauge mapping of Corba OTS. A JTS transaction manager implements the Java mappings of the CORBA OTS 1.1 specification, specifically, the Java packages org.omg.CosTransactions and org.omg.CosTSPortability. A JTS transaction manager can propagate transaction contexts to another JTS transaction manager via IIOP.

This post originally appeared on an external website

by narendrn

A lot of people are familiar with the Tibco RV (Rendezvous) product. This product is around 20 years old and powers many mission critical systems across different domains. Tibco has another product called Tibco EMS (Enterprise messaging Server) which is based on the JMS hub-and-spoke model and is quite popular in EAI. So what's the difference btw the two products? - When to use what ?

First we need to understand the differences in the architecture between the two products. Tibco RV is based on the multicast publish/subscribe model, where as Tibco EMS is based on the hub-and-spoke model.

Multicast is the delivery of information to a group of destinations simultaneously using the most efficient strategy to deliver the messages over each link of the network only once and only create copies when the links to the destinations split. By comparison with multicast, conventional point-to-single-point delivery is called unicast. When unicast is used to deliver to several recipients, a copy of the data is sent from the sender to each recipient, resulting in inefficient and badly scalable duplication at the sending side.

Googled out a white paper that contained some good architecture details about Tibco RV. The link is here.

Excerpt from the article:
TIB/RV is based on a distributed architecture . An installation of TIB/RV resides on each host on the network.Hence it eliminates the bottlenecks and single points offailures could be handled. It allows programs to sendmessages in a reliable, certified and transactional manner,depending on the requirements. Messaging can be delivered in point-to-point or publish/subscribe, synchronously orasynchronously, locally delivered or sent via WAN or theInternet. Rendezvous messages are self-describing and platform independent.
RVD (Rendezvous Daemon) is a background process that sits in between the RVprogram and the network. It is responsible for the deliveryand the acquisition of messages, either in point-to-point or publish/subscribe message domain. It is the most important component of the whole TIB/RV.Theoretically, there is an installation of RVD on every host on the network. However, it is possible to connect to a remote daemon, which sacrifices performance and transparencies. The RV Sender program passes the message and destination topic to RVD. RVD then broadcasts this message using User Data Packet(UDP) to the entire network. All subscribing computers with RVDs on the network will receive this message. RVD will filter the messages which non-subscribers will not be notified of the message. Therefore only subscriber programs to the particular topic will get the messages.

So when to use Tibco RV and when to use Tibco EMS (or any other hub-and-spoke model). An article at http://eaiblueprint.com/3.0/?p=17 describes the trade-offs between the two models:

Excerpt:
In my opinion, the multicast-based publish/subscribe messaging is an excellent solution for near-real-time message dissemination when 1 to 'very-many' delivery capabilities matter. RV originated on the trading floors as a vehicle for disseminating financial information such as stock prices.

However, in most EAI cases the opposite requirements are true:
?Cardinality? of message delivery is 1-1, 1-2; 1 to-very-many is a rare case. With exception of ?consolidated application? integration model (near real time request reply with timeout heuristics), reliability of message delivery takes priority over performance. Effectively, in most EAI implementations of RV, RV ends up simulating a queueing system using its ?Certified Delivery? mechanism. While this works, it is a flawed solution (see the discussion of Certified Messaging below).

Reliable delivery is a ?native? function of hub-and-spoke solutions. The multicast solution must be augmented with local persistence mechanism and re-try mechanism.
While RV offers reliable delivery (queueing) referred to as Certified Messaging, this solution is flawed in that: Inherently, reliability of CM is not comparable to the hub-and-spoke topologies as the data is stored in local file systems using non-transactional disk operations, as opposed to centralized database in hub-and-spoke topology. Corruption of RV ?ledger files? is not a rare case that leads to loss of data.
Temporal de-coupling is not the case. While a message can be queue for a later delivery (in case the target system is unavailable), for a message to be actually delivered both system must be up at the same time.

RV / CM lacks a basic facility of any queueing system: a queue browser that is frequently required for production support (for example, in order to remove an offending message). In contrast, MQ Series offers an out-of the box queue browser and a host of third party solutions. Transactional messaging is difficult to implement in multicast environment. TIBCO offers a transactional augmentation of RV (RVTX) that guarantees a 1-and-onlu-once delivery; however, this solution essentially converts RV into a hub-and-spoke system. Consequently, very few RV implementations are transactional.

Having said that, the shortcomings of hub and spoke include:

  • Single point of failure, when the hub is down, everything is down. MQ Series addresses this problem by providing high-reliability clustering.
  • Non-native publish/subscriber (publish/subscribe is emulated programmatically ) that results in reduced performance, especially in 1-to-very many delivery
  • Overhead of hub management (a need for administering hub objects: queues, channels, etc).
  • Inferior performance, especially in 1 to-many publish/subscribe and request/reply cases.

These shortcomings ? in my opinion ? do not outweigh benefits and for that reason MQ Series / hub-and-spoke solutions constitute a better choice for most EAI problem. Multicast-based publish subscribe is better left to its niches (high volume, high performance, accepter unreliability, 1 to very many).

This post originally appeared on an external website

by narendrn

I was quite familiar with Terminal Services in Windows and also the Remote Desktop Protocol used in it. But recently I came across a technology called 'Citrix' that was similar in nature, so I decided to dig into the root of it. This is the info I found on the net:

Citrix Presentation ServerSoftware from Citrix that provides a timeshared, multiuser environment for Unix and Windows servers. Formerly MetaFrame, Citrix Presentation Server uses the ICA protocol to turn the client machine into a terminal and governs the input/output between the client and server. Applications can also be run from a Web browser
Citrix Presentation Server (formerly Citrix MetaFrame) is a remote access/application publishing product built on the Independent Computing Architecture (ICA), Citrix Systems' thin client protocol. The Microsoft Remote Desktop Protocol, part of Microsoft's Terminal Services, is based on Citrix technology and was licensed from Citrix in 1997. Unlike traditional framebuffered protocols like VNC, ICA transmits high-level window display information, much like the X11 protocol, as opposed to purely graphical information.

Independent Computing Architecture (ICA) is a proprietary protocol for an application server system, designed by Citrix Systems. The protocol lays down a specification for passing data between server and clients, but is not bound to any one platform.
Practical products conforming to ICA are Citrix's WinFrame and MetaFrame products. These permit ordinary Windows applications to be run on a suitable Windows server, and for any supported client to gain access to those applications. The client platforms need not run Windows, there are clients for Mac and Unix for example.

This post originally appeared on an external website

by narendrn

DMZ stands for demilitarized zone. DMZ is also known as perimeter network and is used for security purposes. A DMZ is that part of the network/subnet that sits between the organisations LAN and the Internet. The concept behind creating a DMZ is that m/c's from the Internet and the org's LAN can connect to DMZ, but the DMZ can only connect to the external network - i.e. the Internet.
This allows m/c's hosted in the DMZ to interact with the external network for services such as Email, Web and DNS. So even if a host in the DMZ is compromised, the internal network is still safe. Connections from the external network to the DMZ are usually controlled using port address translation (PAT).

A DMZ can be created by connecting each network to different ports of a single firewall (3-legged-firewall) or by having 2 firewalls and the area btw them as a DMZ.
In case of Enterprise Applications (3-layered), the webserver is placed in the DMZ. This protects the applications business logic and database from intruder attacks.



This post originally appeared on an external website

by narendrn

A lot of my developer friends pop up the question of why to use a webserver when today's application servers such as Weblogic, Websphere, Tomcat, JBoss etc. also have a HTTP listener. But in almost every architecture, we will see Apache in front of Tomcat, or IBM Http Server in front of WAS. The question is why?...and the answers are as follows:

  • Webservers serve static content faster than application servers. Hence for performace reasons, it makes sense to shift all the static contect to the webserver. (J2EE developers hate this, as they now have to split the *.war files)
  • The webserver can be put in a DMZ to enable enhanced security.
  • The webserver can be used as a load balancer over multiple application server instances.
  • The webserver can have agents/plug-in's to security servers such as Netegrity. Hence security is taken care by the webserver, putting less load on the application server.

This post originally appeared on an external website

by jaqui

Apache isn't just a webserver though.
it's a file server, a network server [ strong extranet module ] and even an APPLICATION server in itself.

heck, apache even has a kitchen sink:

Name: apache-mod_kitchensink

Version: 2.0.54_1.1-2
Size: 51 KB

Summary: Mod_KitchenSink is a DSO module for the apache web server

Description: Mod_KitchenSink was inspired by a discussion around Mozilla bug #122411. To make a story short, if people ever say "ADVX.org is too bloated with useless features, it has everything but the kitchen sink", they just have been proved wrong.

With this module, ADVX *has* a kitchen sink. Just install this package, and point your browser to http://localhost/kitchensink.

by barddzen

I have asked the same questions, but the answers you mentioned can be solved in other ways besides plopping Apache in front of your app server. For example, many of the proxy systems today such as Netegrity and Net Scaler also come with a customized and locked down web server already, so hooking your Apache into a system like that only adds yet another web server to the equation.


The other issue with another web server on top of app server is products like Netegrity means the app servers have to assume the user has already been authenticated and looks for elements in the http header, so anyone passing these values directly to the app server can simply bypass the proxy and get to everything anyway, so how does this add to security?


How is having static content, images, and flat HTML in a DMZ more secure? In fact, who gives a crap if someone has access to that content anyway? Anyone who knows how to right-click, view source can get to it and there are a bunch of tools that can suck entire web sites: The point being, securing flat and static HTML is this really that big of a security concern for most of us? Maybe in some instances, but the vast majority of sites?  I don't think so.  I'd rather concentrate on securing everything the app serve does and secure that layer more since it deals with compiled code, data connections, etc.

Splitting .war files is just a headache in and of itself and I see no point in doing that. Depending on the middle hardware/proxy you get, most devices now a days can be setup to cache common requested static content out of the box so complicating your deployment process just for the sake of doing it isn't even worth the effort in my mind. Let the tools do their job.

Just my 50 cents...

by narendrn

Recently I had to move my MySQL server from a WinXP machine to a Win2000 server machine. I thought that the data migration of the MySQl database would be a pain.

To my suprise, it just took me 5 mins flat to do it. Here are the steps:
1)First export the database into a flat file. This exports not just the schema but also the "insert" statements for all the data in the tables
mysqldump -u DBUSER -p DBNAME > DBNAME.sql
substituting DBUSER with your MySQL username and DBNAME with your database name.
2) Create a new empty schema/database on your new machine. The name of the database will probably be same as the name in the old server.
3) Import the dump file into the new database server.
mysql -u DBUSER -p -h MACHINE-NAME DBNAME<>
substituting DBUSER with your MySQL username and DBNAME with your database name, and MACHINE-NAME with the name of the SQLServer instance - quite often the name of the machine itself.

That's it...migration of database done.

This post originally appeared on an external website

by narendrn

Virtual hosting is essentially "shared web hosting". Virtual hosting is a method that webservers use to host more than one domain name on the same host. It is generally the most economical option for hosting as many people share the overall cost of server maintenance.

There are two basic methods of accomplishing virtual hosting:
name-based, and IP address or ip-based.

In name-based virtual hosting, also called shared IP hosting, the virtual hosts serve multiple host names on a single machine with a single IP address.
With web browsers that support HTTP/1.1 (as almost all do today), upon connecting to a webserver, the browser sends the URL to the server. The server can use this information to determine which web site to show the user. So each website would have a DNS entry and all the DNS entries for the websites point to the same IP address.

In IP-based virtual hosting, also called dedicated IP hosting, each webserver has a different IP address. The webserver is configured with multiple physical network interfaces, or virtual network interfaces on the same physical interface. The webserver uses the IP address from the HTTP request to determine which web site to show the user.

Disadvantages of virtual hosting:

Older web browsers that only support HTTP/1.0 will not work because they do not send the URL.
If the Domain Name System (DNS) is not properly functioning, it becomes much harder to access a virtually-hosted website.
They do not support secure websites (HTTPS), at least not on the same TCP port. All virtual hosts on a single HTTPS webserver must share the same digital certificate. Because the SSL handshake takes place before the expected hostname is sent to the server, the server doesn't know which encryption key to use when the connection is made.

This post originally appeared on an external website

by narendrn

Recently I was trying to evaluate some open-source portals that I can use for my personal use.
Here is the list of interesting portals I found out:

http://www.exoplatform.com/

http://www.liferay.com/

http://portals.apache.org/

http://jboss.com/products/jbossportal

http://www.gridsphere.org/

Out of the above I found Apache Jetspeed 2.0 and Liferay to be promising...I had a bad experience with JBoss Portal...it sucks !!!

This post originally appeared on an external website

by narendrn

Recently I developed a portal which made use of MySQL database. The database was small in size and had only a few tables. But I was facing problems, when the web-application had to be installed in different locations. I had to give instructions to users on how to create the MySQL database, run the script to add all the necessary tables etc.

Then I came across HSQL (Hypersonic SQL) database. This is a 100% pure Java database and can be run in-process inside a Java application...even a webapplication. No need to install anything or start any database process. More information about HSQL can be found at http://www.hsqldb.org/

For me, just the following 2 lines of code, created a database in my Tomcat webapps directory.
Class.forName("org.hsqldb.jdbcDriver");

Connection c = DriverManager.getConnection("jdbc:hsqldb:file:webapps/myPortal/database/narenDB", "sa", "");

So simple.........:)

This post originally appeared on an external website

by pkolega

HSQL has been around for, well... forever. Yes, it's amazing little database, very well suited for Java development. As for Derby/Cloudscape being "lite DB/2" -- simply not true. Yes - the JDBC drivers you use to connect to them are the same, but underlying technology is very much different. Cloudscape came to IBM when they took over Informix; IBM turned codebase of it (after some cleanup, I'm pretty sure...) to Apache. But I think they still sell and support Cloudscape as a product. BTW: being able to use the same drivers to connect to Cloudscape and DB/2 is a nifty trick - you can use one for development and the other for deployment -- "just-change-connection-string". And yes, I know - it may not be _that_ easy.

Peter W.

by narendrn

Pooling basically means utilizing your resources better. For example, imagine a fairly large
number of clients utilizing a small number of database or network connections efficiently. By
limiting object access to only the period when the client requires it, you can free resources for
use by other clients. Increasing utilization through pooling usually increases system
performance.

You can use pooling to minimize costly initializations. Typical examples include database and
network connections and threads. Such connections often take a significant amount of time to
initialize. You can achieve significant savings by creating these connections once and reusing
them later. Finally, you can pool objects whose initialization is expensive, in terms of time,
memory, or other resources. For example, most containers pool Enterprise JavaBeans to avoid
repeated resource allocation and state initialization.

Examples of objects holding external OS resources:
-database connection pooling,
- socket connection pooling (including HTTP, RMI, CORBA, WS, etc...)
- thread pooling,
- bitmaps, fonts, other graphics objects...

For objects that do not hold external OS resources, but are time-consuming to initialize and are
memory-heavy, a cost benefit analysis needs to be done. What is the cost of pooling
(synchronization, management, GC etc.) vs the benefit (quick access to connection or socket).One very important concern during pooling is the problem of concurrency bottlenecks....when we snchronize access to the pooled resources.

Simple object pooling (which hold no external resources, but only occupy memory) could be a
waste, as the performance improvments in the latest versions of the JVM have tilted this
equation, so we better re-evaluate what we think we know........There was also an old school of thought that pooling reduces the need of GC and hence improves the performance. But with the introduction of the concept of 'generations' to GC algorithms, it is possible that pooled resources would be more difficult to GC as they are put in older generations of objects.

But for external resources the JVM improvements are of no-concern. Whether objects holding
external resources (out of JVM, either OS resources on JVM host machine, or on the other machine) should be pooled or not, does not depend on JVM efficiency, but on their availability, weight and and other costs of holding these resources.

Why stateless session beans pooled ?The EJB spec states that only one thread at the time can execute method code in a statelss session bean instance. So you need to pool them to provide concurrency. The qustion is whether this spec requirement makes sense?
As only reason for this request I can see is that this way you can have local instance state (as
instance fields) in a stateless session bean that is valid during a single method execution. Note
that your method can call other (possibly private) methods of the same bean, so in this scenario
this makes sense.
However, you can get the same behavior by holding a state and logic in a separate POJO and having your session bean playing a facade to that POJO. Probably this request is to simplify programming model for stateless session EJBs, making them completely thread safe .
The EJB standard does not require that Stateless Session EJBs have no internal state (ie.
fields). It simply requires that no conversational state be maintained between the bean and its
clients. A pool of these that are doled out to client threads on a per-call-basis is valuable if
the bean manages some expensive-to-allocate-or-create resource.

From the EJB 2.0 Specification, section 7.5.6: "A container serializes calls to each session bean instance. Most containers will support many instances of a session bean executing concurrently; however, each instance sees only a serialized sequence of method calls. Therefore, a session bean does not have to be coded as reentrant."

Is there some other reason why stateless session beans are pooled?...Something in the
architecture of EJB that forces this decision?

This post originally appeared on an external website

by narendrn

We often face the problem of retrieving a 'auto-generated' column after a 'INSERT' operation. For e.g.

insert into myTable (name) values ('Naren')");

Now immediately after the insert, I want to get the 'auto-generated' key of the inserted row. The only way to do it earlier in JDBC was to fire another query to get the last inserted row. But this method is fundamentally flawed, as there could be other inserts that have happened in-between.

Now, in JDBC 3.0 (in JDK 1.4), we have core JDBC methods that enable us to retrive the autogenerated key automatically. For e.g.

int rowcount = stmt.executeUpdate("sql here...",Statement.RETURN_GENERATED_KEYS);
ResultSet rs = stmt.getGeneratedKeys ();

The ability to retrieve auto generated keys provides flexibility to the JDBC programmer and it provides a mechanism to realize performance boosts when accessing data.

The following methods are available since JDK 1.4

public int executeUpdate(String sql, int autoGeneratedKeys) throws SQLException

Executes the given SQL statement and signals the driver with the given flag about whether the auto-generated keys produced by this Statement object should be made available for retrieval.
Parameters:
sql - must be an SQL INSERT, UPDATE or DELETE statement or an SQL statement that returns nothing.
autoGeneratedKeys - a flag indicating whether auto-generated keys should be made available for retrieval; one of the following constants: Statement.RETURN_GENERATED_KEYS
Statement.NO_GENERATED_KEYS

---------------------------------------

public int executeUpdate(String sql, int[] columnIndexes)
throws SQLExceptionExecutes the given SQL statement and signals the driver that
the auto-generated keys indicated in the given array should be made available for retrieval. The driver will ignore the array if the SQL statement is not an INSERT statement.

Parameters:
columnIndexes - an array of column indexes indicating the columns that should be returned from the inserted row

------------------------------------------

public int executeUpdate(String sql, String[] columnNames)
throws SQLExceptionExecutes the given SQL statement and signals the driver that
the auto-generated keys indicated in the given array should be made available for retrieval. The driver will ignore the array if the SQL statement is not an INSERT statement.

Parameters:
columnNames - an array of the names of the columns that should be returned from the inserted row

----------------------------------------

In Spring JDBC API, the JDBCTemplate has a utility method that directly gives the genetated key:

int update(PreparedStatementCreator psc, KeyHolder generatedKeyHolder)

This post originally appeared on an external website

by narendrn

I have noticed that a lot of people are wasting their precious time changing code that contains "+" overloaded String operator toStringBuffer...in the following way :
For eg.
-----------------
1 ) String sTemp = "a" + "b" + "c";is being replaced with
---------------
2 )StringBuffer sBuffer = new StringBuffer()

sBuffer.append("a");

sBuffer.append("b");

sBuffer.append("c");

String sTemp = sBuffer.toString();

Now the point is for simple string concatenations such as the above the compiler itself does the conversion to StringBuffer. Thus the code in 1) gets converted to code in 2) during compilation time only.......ie conversion to platform independant byte form...

So there is no need for us to do this explictly....But yes, there will be a remarkable advantage when we have String concatenation in a for loop...
for eg, if the above code was within a for loop.....for eg:
for(int i=0;i<n;i++)
{ sTemp = "a" + "b" + "c";}

Then it makes sense to use StringBuffer, otherwise a new String Buffer object will be created for each iteration of the loop. So in such cases we can instantiate the StringBuffer outside the loop and just use "append" inside the loop..

This post originally appeared on an external website

by narendrn

For faster startup access to a website, it is sometimes desirable to precompile all the JSP pages on a production server, so that the first users of the website do not have to bear the performance hit of JSP compilation (first to servlet java file and then to class file)

Tomcat 5.0 and above comes with a cool ANT script that uses the Jasper JSP compiler to precompile the JSPs in a webapp. The ANT script is available here.

On running the ANT script, the "WEB-INF\classes" directory has a new package "\org\apache\jsp" that contains all the class files of the compiled JSPs. The src folder would contain the generated Java Servlet files.

If U edit a JSP page on the fly, then it would again be compiled at run-time and swapped in memory with the old one, but the classes directory would still contain the old class file.

This post originally appeared on an external website

by narendrn

JPDA stands for Java Platform Debugger Architecture. It is a new debugging design adopted by the VM that facilitates debuggers to attach to the VM process easily. More info about JPDA can be found here.

I tried out debugging a Struts application running on Tomcat using Eclipse. (Versions: Tomcat-5.0; Eclipse-3.1.2)
Here are the steps to be followed:

  1. Start Tomcat in JPDA debug mode. For this, set up two variables in the environment as follows:
  2. The first variable is: set JPDA_TRANSPORT=dt_socket (This tells the VM that the debugger would talk to the VM on Tcp sockets)
  3. The second variable is: set JPDA_ADDRESS=8000 (This tells the VM the port on which it should listen for debugger connections)
  4. Go to the command prompt and run the 'Catalina.bat' batch file present in {TomcatHome}\bin. On the command prompt pass the following argument: catalina jdpa start
  5. Go to Eclipse and open the Debug window. Create a new configuration under 'Remote Java Application' . Make sure 'Connection Type' is Socket and port is 8000
  6. Navigate the webapplication thru web browser and see Ur Eclipse stop at breakpoints..

Happy debugging............

This post originally appeared on an external website

by narendrn

I was trying to get the hang of AJAX and was learning a lot of JavaScript and HTML. I was confused btw the <div>and <span> tags in HTML. A quick google came up with these answers:

The main difference between <div> and <span>is that <div>defaults as a block-level element and <span>defaults as an inline element.
Block-level elements create a new formatting context. They are stacked vertically(acts as a paragraph break) in the order they appear in the source code and may contain any number of other block-level elements or inline elements.
Inline elements do not create a new formatting context. They line up horizontally within their block container in the order they appear in the source code, wrapping to new lines within their block container if necessary and can contain only other inline level elements (or inline text boxes).
Both <div> and <span>are generic elements. Both can be used as many times as you like on a page. They hold no defaults settings besides their value for display (block on <div>, inline on <span>)

This post originally appeared on an external website

by narendrn

While working with VisualStudio.NET, I was impressed with the ability of VS.NET to even debug JavaScript on a HTML page. It saved a lot of time - or the other alternative was writting alert() statements.

I wondered if there was some similar JavaScript Debugger tool for the Java community.
Fortunately, Mozilla has come up with a cool tool that allows developers to debug Javascript code. The tool is "Venkman Debugger" and it's a free download.

Here's the URL for more details: http://www.mozilla.org/projects/venkman/

For firefox 1.0 (my fav browser), the debugger is available as a plug-in. Installation was a breeze !!!
I just had to download the *.xpi" file and drag and drop the file in a open window of Firefox...that's it..Then start the debugger from the tools menu...

After this, any page U see on Firefox - the debugger windows will show all the Javascript functions and we can set breakpoint and step thru the code...also available is the local window and watch window :)

Happy debugging..

This post originally appeared on an external website

by narendrn

Recently came across a wonderful blog stating a few things about Unix/Linux which ever developer should know. The link to the blog is here.
Some useful points in the blog were:

  1. In Unix, if U want Ur application or server to use a port less than 1024, then U must be running as 'root'. This imposes security risks !!!
  2. In Unix, it is possible for one process to open a file stream and an another process to delete the same file. (Windows does a automatic file lock). For the same reason, in Unix, U can delete a directory right from under it.
  3. Equivalent of Windows services in the Unix world are just plain shell scripts that are mentioned in the /etc/init.d directory.
  4. The Unix shell runs scripts by creating a copy of itself and running the script in the new shell. This new shell will read in the script, perform all the steps in the script (e.g. set all the environment variables) and then exit, leaving the original shell and its environment unchanged. So to run scripts in the same shell, we need to type ". setenv.sh" For c-shell, we need to type "source setenv.sh"
  5. And last, but not the least, every developer learns early - Unix new line character is \n and on Windows \r\n.

This post originally appeared on an external website

by narendrn

We are all aware of proxy servers, that enable us to access the internet from inside a firewall. These proxies are known as forward proxies. Similarly we have reverse proxy that sits in front of a webserver and can provide caching and also redirection.

Recently, we needed to expose a website to the internet. I thought the only way of doing it would be to assign a new IP to the webserver machine and have a DNS setup for it. Alternatively, we can put in one more NIC card and have that IP address exposed to the public.

But Reverse Proxy presents us with an interesting alternative. A reverse proxy can be used to enable controlled access from the Web at large to servers behind a firewall.
Here is a simple example provided at the Apache website (Apache can also be used as a reverse proxy server)

Scenario:
Company example.com has a website at www.example.com, which has a public IP address and DNS entry can be accessed from anywhere on the Internet.

The company also has a couple of application servers which have private IP addresses and unregistered DNS entries, and are inside the firewall. The application servers are visible within the network - including the webserver, as "internal1.example.com" and "internal2.example.com", But because they have no public DNS entries, anyone looking at internal1.example.com from outside the company network will get a "no such host" error.

A decision is taken to enable Web access to the application servers. But they should not be exposed to the Internet directly, instead they should be integrated with the webserver, so that http://www.example.com/app1/any-path-here is mapped internally to http://internal1.example.com/any-path-here and http://www.example.com/app2/other-path-here is mapped internally to http://internal2.example.com/other-path-here. This is a typical reverse-proxy situation.
--------------------------------------------------
In my scenario, my network admin guys were able to put my website on the internet without touching my server even once :)

For more information on reverse proxy check out the following links:
http://www.apacheweek.com/features/reverseproxies
http://www.informit.com/articles/article.asp?p=169534&rl=1
http://en.wikipedia.org/wiki/Reverse_proxy

This post originally appeared on an external website