Batch RPCs for GWT

We had developed GWT beginning from 2009. Framework was becoming more popular.
Up to a point it was like a dream framework. It was like developing
windows application on browser. It is really capable of whatever you want to.
If you have knowledge in Swing ,it is similar and productivity is unbelievable.

But when project grows, to 200+ pages it became more and more slow.
There were lots of optimizations. We did nearly all of them.
But it was not possible to come up with lightweight product.

I will write small posts with useful ideas about GWT.


Think you have a page which includes independent 10,20 components.
When page is loaded all of them are making their own RPC. This adds big delays
on page loads.

Idea is,do not call RPC wait for 500 milliseconds batch them and call them together.

protected ArrayList<YRPCRequest> reqQueue = new ArrayList<YRPCRequest>( ) ;

    protected ArrayList<AsyncCallback<YRPCResponse>> cbQueue = new ArrayList<AsyncCallback<YRPCResponse>>( ) ;

protected YRPCTimer timer = new YRPCTimer( 500 )
    {
        @Override
        public void run( )
        {
            fireRegisteredRPCs( ) ;
        }
    } ;
 
 
 protected void registerCallImpl( YRPCRequest req, AsyncCallback<YRPCResponse> callback, boolean isCached )
    {
        final long time = System.currentTimeMillis( ) ;
        

       
        this.reqQueue.add( null ) ;
        

        this.cbQueue.add( callback ) ;

        
    }

 
 @RemoteServiceRelativePath( "YRPCServiceGateway" )
public interface IYRPCServiceGateway extends RemoteService 
{
    public static class Util
    {
        public static IYRPCServiceGatewayAsync getInstance( )
        {
            return GWT.create( IYRPCServiceGateway.class ) ;
        }
    }

    public ArrayList<YRPCResponse> serviceMultiplexer( ArrayList<YRPCRequest> reqList ) ;
}


@Override
    public void onSuccess( ArrayList<YRPCResponse> resList )
    {
        lastResponse = resList ;
        try
        {
            long totalServerTime = -1 ;
            for( int ii = 0; ii < this.cbQueue.size( ); ++ii )
            {
                final int i = ii ;

                final YRPCResponse res = i < resList.size( ) ? resList.get( i ) : null ;

    
    
    Scheduler.get( ).scheduleDeferred( new Scheduler.ScheduledCommand( )
                    {
                        private AsyncCallback<YRPCResponse> callback = GwtRpcGateway.this.cbQueue.get( i ) ;

                        @Override
                        public void execute( )
                        {
                            this.callback.onSuccess( res ) ;
                        }
                    } ) ;
            }


            this.reqQueue.clear( ) ;
            this.reqQueue = null ;
            this.cbQueue.clear( ) ;
            this.cbQueue = null ;
        }
        catch( final RuntimeException e )
        {
            e.printStackTrace( ) ;
            throw e ;
        }
        finally
        {
            if( this.isBlocked )
            {
                // YComponentUtils.enableBodyComponent( true ) ;
            }
        }

    }

Corrective Action for Hibernate

An undo functionality is necessary for all systems. When customer calls for a bad data
that he created he is not willing to cancel it. He wants to delete all evidences
from system.

Think we are creating a purchase order. We can cancel order. But user wants to delete
created products ,shipment and payment terms...

So what can we do is refer to article for "Event Loggging".
We need to find the session id of main object.(Purchase Order)
And find all the objects related with it.
Then delete.

Right?
What about updates done to Purchase Order?

So we need a field on our beans that creates and updates our beans.
Open a field concatenate every session id that affected your bean.
1st item will be creator sessionid,others will be updaters.

Then a query delete all related items will be like :

SessionId Concatanted : 12,34,46
12 : creator session 34,46 updaters

delete from tableX where sessionid in (12,34,46);
delete from tableY where sessionid in (12,34,46);
delete from tableZ where sessionid in (12,34,46);

Code for this part is easy if you read other thread.
It is important to get the idea.

Of course one can do this with JDBC. We only need to have unique number(or string)
for every transaction we are doing.

Here I say session usually. Because our sessions are single transaction and atomic.
So while reading you can realize to call this transaction.It depends on your model.

Using GWT BaseModelData object for infinitive flexibility

We have implemented a project with GXT( a GWT framework) and Hibernate.

It has a class like this.
com.extjs.gxt.ui.client.data.BaseModelData

Idea is put fields into Hashmap and read them by key. This object goes between
server and browser.

public User (

 protected Organization organization;
 protected String name;
 protected String surname;
 
 }
 
 //will be 

public UserDao extends  BaseModelData( 
 
    //Utility methods for explicit access
    public Organization getOrganization( ) 
    {
        return this.< Organization >get( "organization" ) ;
    }

    public String getName( ) 
    {
        return this.< String >get( "name" ) ;
    }
    
    public String getSurname( ) 
    {
        return this.< String >get( "surname" ) ;
    }
    
    

So what is so flexible about this?

We found this very useful when creating a sample bean.
We have everything in map. So for organization field we can even put
an organization or List.

If you check Database querying by Object notation article you will understand this is for.

User sample = new User();
user.set( "organization",new Organization[]{org1,org2,org3});
user.set( "name",new Organization[]{"John","Jack"});

I can send this sample Object and interpret this into a query where
Bring users in any of organizations (org1,org2,org3) having name either John or Jack.

Object obj = sample.get("organization");

if( obj instanceof Organization )
{

}else if( obj instanceof List)
{

}

Think, how will you make this without this model?

Hibernate Audit By keeping order of change

There are lots of samples in internet creating audit data for hibernate with Interceptors.

With these samples (maybe I missed some of them) you do not know the order of changes.
We needed to know the order for some reasons.

Luckily our hibernate code was generated with a tool we wrote. We put a method as 1st
line in our setter methods. We were calling a method to understand if this field is changed.
(By this way u can also get full path (stacktrace) of change )

public class User {
 public void setPersonel( YPerson personel )
    {
        this.setEditedFieldValue( "personel", this.personel, personel, true ) ; 
        this.personel = personel ;
    }
 
 public void setCode( String code )
    {
        this.setEditedFieldValue( "code", this.code, code, true ) ; 
        this.code = code ;
    }
 
 }

 

Below is the generic code to check if field is changed.
There are cases like
1)Object created with new,we are assigning necessary fields (You do not need to keep ,
because objects former values are null before creation so if an object does not have any audit data,it is same from beginning)
2)Object is being read from database ( We are attaching object so all values will seem null,be careful)
Hibernate creates object with null constructor and calls setters.
3)Object attached , in the flow we changed a field.

protected void setEditedFieldValue( String fieldName, Object oldValue, Object newValue, boolean isNullable )
    {
        //Beans can override isIntercepted method to skip being audited
        if( !this.isIntercepted( ) )
        {
            return ;
        }
        
        //Do not need to audit initially null ones, or you can skip at time of attaching
        if( oldValue == null )
        {
            return ;
        }

        //for not null int values our code has 0. So if something is not nullable (we know at time of code generation )
        // but it is 0 it means null for us return without auditing
        if( !isNullable )
        {
            if( ( oldValue instanceof Integer ) && ( ( Integer )oldValue ).intValue( ) == 0 )
            {
                return ;
            }

            if( ( oldValue instanceof BigDecimal ) && ( ( BigDecimal )oldValue ).compareTo( BigDecimal.ZERO ) == 0 )
            {
                return ;
            }
        }

        //Do not audit if value did not change,
        //Example : You opened edit page, change some values and some not.So audit only changed ones.
        if( oldValue == newValue )
        {
            return ;
        }

        

        if( oldValue instanceof YBean )
        {
            String oldDisp = "[N/A]" ;
            String newDisp = "[N/A]" ;

            if( YServerConstants.IS_AUDIT_DEBUG_MODE )
            {
                //want to know when if beans are proxy or no.
                    if( HibernateUtils.isproxy( ( YBean )oldValue ) )
                    {
                        oldDisp = "##BeanProxy##;" + oldValue.getClass( ).getSimpleName( ) + "; id:" + ( ( YBean )oldValue ).getId( ) ;
                    }
                    else
                    {
                        oldDisp = "##YBean##;" + ( ( YBean )oldValue ).getInstanceBeanName( ) + "; id:" + ( ( YBean )oldValue ).getId( ) + "; disp:" + ( ( YBean )oldValue ).getShortDisplay( ) ;
                    }

                    if( newValue != null )
                    {
                        if( HibernateUtils.isproxy( ( YBean )newValue ) )
                        {
                            newDisp = "##BeanProxy##;" + newValue.getClass( ).getSimpleName( ) + "; id:" + ( ( YBean )newValue ).getId( ) ;
                        }
                        else
                        {
                            newDisp = "##YBean##;" + ( ( YBean )newValue ).getInstanceBeanName( ) + "; id:" + ( ( YBean )newValue ).getId( ) + "; disp:" + ( ( YBean )newValue ).getShortDisplay( ) ;
                        }
                    }
                

                if( !oldDisp.equals( newDisp ) )
                {
                    if( this._editedFieldValuesPool == null )
                    {
                        this._editedFieldValuesPool = new YAuditFieldNameValuePairPool( ) ;
                    }

                    this._editedFieldValuesPool.setFieldValue( fieldName, oldDisp ) ;
                }
            }
        }
        else
        {
            //We do not know if any of objects are null so can not call , o1.equals(02) 
            if( isObjectEquals( oldValue, newValue ) )
            {
                return ;
            }

            if( this._editedFieldValuesPool == null )
            {
                this._editedFieldValuesPool = new YAuditFieldNameValuePairPool( ) ;
            }

            this._editedFieldValuesPool.setFieldValue( fieldName, oldValue ) ;
        }
    }
    
    
    public static boolean isObjectEquals( Object o1, Object o2 )
    {
        if( o1 == o2 )
        {
            return true ;
        }

        if( o1 == null || o2 == null )
        {
            return false ;
        }

        return o1.equals( o2 ) ;
    }

    
    
    //This class is holding audit data. Since audits are hold  at Arraylist you do not loose order.
    //Also you can even add stack traces with conditions to monitor your system.
    
    
    
    
public class YAuditFieldNameValuePairPool implements Serializable
{
    protected ArrayList<String> nameList = new ArrayList<String>( 0 ) ;

    protected ArrayList<Object> valueList = new ArrayList<Object>( 0 ) ;
    
    
    public int setFieldValue( String fieldName, Object value )
    {
        boolean isFound = false ;

        //check If you have same record do not save again
        //This is a decision you can think this is usefull to see if
        //a change is being applied mutiple time.
        for( int i = this.nameList.size( ) - 1; i >= 0; --i )
        {
            String name = this.nameList.get( i ) ;
            Object val = this.valueList.get( i ) ;

            if( name.equals( fieldName ) )
            {
                if( YClientUtils.isObjectEquals( value, val ) )
                {
                    isFound = true ;
                    break ;
                }
            }
        }

        if( !isFound )
        {
            this.nameList.add( fieldName ) ;
            this.valueList.add( value ) ;

            return this.nameList.size( ) - 1 ;
        }

        return -1 ;
    }

An alternative to Hibernate Criteria and HQL

An alternative to Hibernate Criteria, Stay in object in notation for querying database,Express your query with object relations.

*** Below logic is like ,
"org.hibernate.criterion.Example"
But more advance and also good for omni searches in all fields of beans.(String fields)

When you use hibernate Example you still define projections,criteria...
Below notation is pure object notation.



We can use HQL or criteria api for querying database. This is hibernate and we are dealing with objects.
We taught why not query database by objects.(Probably lots of people did this by time
when they advance in hibernate)
For example :
I want users with name John having organization 'ABC Company'

hql = ' Select dao FROM User dao where dao.name like '%John%' and dao.organization.name like '%ABC Company%' "

Criteria


Full Objecte Notation
Create detached object representing what you look for
User sample = new User();
sample.setName("John");

Organization org = new Organization();
org.setName("ABC Company");
sample.setOrg(org);

If you have builder pattern

(New User()).setName("John").getOrganization().setName("ABC Company");

This is abusing Hibernate or Just going up one level in Object notation.
Express your query with object relations.

Of course this is translated into HQL or Criteria for querying. We wrote both versions. They both seemed very nice to us.
We love our this piece of code so much :) We wrote this at 2009. At that time there was nothing like this(as we search).
Maybe now there is.

If ultimate goal of Hibernate is to stay in Object notation and ease of use, I think a notation like this
must be alternative like HQL or Criteria.


Below code enables deep search on multiple beans and fields also.

--- Find relations for this user
--- Hql equivalent = " Select dao FROM YRelUserHorg dao where dao.user.id = this.id "
YRelUserHorg beanExample = new YRelUserHorg( ) ;
beanExample.setUser( this ) ;
List beanList = HibernateUtils2.execOmni( null, beanExample, "%", 0, null ) ;

--Discussion is a self referencing table
--Find replies to this thread
--- Hql equivalent = " Select dao FROM YDiscussion dao where dao.reply.id = " + this.mainDiscussion.getId( ) ;
YDiscussion sample = new YDiscussion( ) ;
YDiscussion reply = new YDiscussion( ) ;
reply.setId( this.mainDiscussion.getId( ) ) ;
sample.setReply( reply ) ;

List beanList = HibernateUtils2.execOmni( null, sample, null, null, null ) ;

--- This is a nice one,u have a bean with a sub bean say u want to search "xyz" on any files
--- In our database everyone working in company is Person and system users are User so, there is a person field in User
--- Hql equivalent = " Select dao FROM YUser dao
-- where dao.title like '%xyz%' or dao.address1 like '%xyz%' or dao.person.name like '%xyz%' or dao.person.surname like '%xyz%' .... think this is
much more(and it really is)
YUser beanExample = new YUser( ) ;
beanExample.setPersonel( new YPerson( ) ) ;

List beanList = HibernateUtils2.execOmni( null, beanExample, "%" + query + "%", 0, this.loadUserItemCount ) ;

--- users having role admin(an object refernce attached object) and having John in any of fields
YRelUserHorg beanExample = new YRelUserHorg( ) ;
beanExample.setRole( adminRole ) ;
bean.setUser(new YUser());


List beanList = HibernateUtils2.execOmni( null, beanExample, "%John%", 0, null ) ;


Of course this code is not full and needs more features but it's talents are really impressive.
Deep searching,and staying at Object notation makes us feel doing something elegant. Because we express our need
with object notation. we can change implementation any time(even to JDBC).

@Transient
    @Override
    public YRelOrganizationOrganization getCriteriaTemplate( String key )
    {
        if( HibernateUtils2.TKEY_MYSUPPLIERS.equals( key ) )
        {
            YRelOrganizationOrganization roo = new YRelOrganizationOrganization( ) ;
            roo.setRelationType( DRelOrganizationOrganization.TYPE_BUYER_SUPPLIER ) ;
            roo.setOtherOrganization( new YOrganization( ) ) ;

            return roo ;
        }
                YRelOrganizationOrganization beanExample = new YRelOrganizationOrganization( ) ;
        beanExample.setStatus( 1 );
        //beanExample.setOtherOrganization( new YOrganization( ) );

        //List<Integer> beanList = HibernateUtils2.execOmni( HibernateUtils2.TKEY_MYSUPPLIERS, beanExample, query + "%", 0, 30, "otherOrganization.interchangeUser.personel.name", SortOrder.ASCENDING ) ;
        List<Integer> beanList = HibernateUtils2.execOmni( HibernateUtils2.TKEY_MYSUPPLIERS, beanExample, query + "%", 0, 30, "otherOrganization.name", SortOrder.ASCENDING ) ;

        
            
            
            
                        if( "String".equals( simpleTypeName ) )
            {
                Object val = subBean.getFieldValue( fieldName ) ;

                if( subBean.query_isForceIncludedField( fieldName ) || ( ( val == null && !IGNORE_IN_SEARCH.equals( val ) ) && !subBean.query_isForceExcludedField( fieldName ) ) )
                {
                    disjunction.add( Restrictions.like( ( alias != null ? alias + "." : "" ) + fieldName, omni ) ) ;
                }
            }
            
                        if( "String".equals( simpleTypeName ) )
            {
                if( subBean.getFieldValue( fieldName ) != null )
                {
                    String val = ( String )subBean.getFieldValue( fieldName ) ;

                    if( subBean.query_isForceIncludedField( fieldName ) || ( ( !YClientUtils.isBlankTrim( val ) && !IGNORE_IN_SEARCH.equals( val ) ) && !subBean.query_isForceExcludedField( fieldName ) ) )
                    {
                        criteria.add( Restrictions.like( ( alias != null ? alias + "." : "" ) + fieldName, val ) ) ;
                    }
                }
            }
            

Logic : we can have template beans (like you are using a type of user select query so much ,so create a template) and also can send an override beanExample
Think like this,
You have a template of users having type Manager this is templateBean
Then you want ones with name Max, you send a sample with name Max.

You can also use this with null template and a sample with both name Max and type Manager

public static List<Integer> execOmni( String key, YBean beanSend, String somnis, Integer firstResult, Integer maxResult, String sort, SortOrder order, boolean isForceOrgId )
    {

        String omnis = YClientUtils.notNullTrim( somnis, "" ) ;

        if( "%%".equals( omnis ) )
        {
            omnis = "" ;
        }

        YBean templateBean = key == null ? null : beanSend.getCriteriaTemplate( key ) ;

        if( templateBean == null )
        {
            templateBean = beanSend ;
        }

        HashMap<String, Criteria> aliasSet = new HashMap<String, Criteria>( ) ;

        Criteria criteria = HibernateManager.getSession( ).createCriteria( beanSend.getClass( ) ) ;

        // Some beans are only readable by their related organizations so a default organization is added every query
        // Some beans are shared like product. A buyer can see supplier product so isForceOrgId is for omitting this.
        if( isForceOrgId && !YServerSession.isTopAdmin( ) )
        {
            if( !( beanSend instanceof YOrganization ) && !( beanSend instanceof YCatalogDetail ) )
            {
                String organizationField = getOrganizationField( beanSend ) ;
                if( organizationField != null )
                {
                    criteria.add( Restrictions.eq( organizationField + ".id", YServerSession.getSession( ).getOrganizationId( ) ) ) ;
                }
            }
        }

        String[ ] beanName = beanSend.getBeanNames( ) ;

        //ADD EXACT VALUES OF BOTH IF THEY ARE NOT EQUAL
        addExactValues( criteria, templateBean, null ) ;
        if( !beanSend.equals( templateBean ) )
        {
            addExactValues( criteria, beanSend, null ) ;
        }

        //Check sub beans ,and if they have value add exact values there
        // (new User()).getOrganization().setName("ABC");
        for( String s : beanName )
        {
            YBean subBean = ( YBean )beanSend.getFieldValue( s ) ;
            //-------------- ADD  EXACT VALUES FROM SEND BEAN
            if( subBean != null )
            {
                String subAlias = "dao_" + s + "_" ;
                if( aliasSet.get( subAlias ) == null )
                {
                    Criteria sub = criteria.createCriteria( s, subAlias, CriteriaSpecification.LEFT_JOIN ) ;
                    aliasSet.put( subAlias, sub ) ;
                }

                addExactValues( criteria, subBean, subAlias ) ;
            }

            //-------------- ADD  EXACT VALUES FROM TEMPLATE BEAN
            subBean = ( YBean )templateBean.getFieldValue( s ) ;
            if( subBean != null )
            {
                String subAlias = "dao_" + s + "_" ;
                if( aliasSet.get( subAlias ) == null )
                {
                    Criteria sub = criteria.createCriteria( s, subAlias, CriteriaSpecification.LEFT_JOIN ) ;
                    aliasSet.put( subAlias, sub ) ;
                }

                addExactValues( criteria, subBean, subAlias ) ;
            }
        }

        // Send your omni query it will be AND or ORs
        //like u write "Europe Max "
        // ( a like '%Europe%' OR b like '%Europe%' ) AND ( a like '%Max%' OR b like '%Max%' ) 
        if( omnis != null && !YClientUtils.isBlankTrim( omnis ) )
        {
            String[ ] splits = omnis.split( " " ) ;

            for( int i = 0; i < splits.length; i++ )
            {
                String split = splits[ i ] ;

                if( YClientUtils.isBlankTrim( split ) )
                    continue ;

                split = split.trim( ) ;

                if( !split.startsWith( "%" ) )
                    split = "%" + split ;

                if( !split.endsWith( "%" ) )
                    split = split + "%" ;

                Disjunction disjunction = Restrictions.disjunction( ) ;

                criteria.add( disjunction ) ;

                setOmniValues( disjunction, templateBean, split, null ) ;
                setOmniValues( disjunction, beanSend, split, null ) ;

                //Do this for also not null sub beans ( going deeper !!! )
                for( String s : beanName )
                {
                    YBean subBean = ( YBean )beanSend.getFieldValue( s ) ;
                    if( subBean != null )
                    {
                        String subAlias = "dao_" + s + "_" ;
                        setOmniValues( disjunction, subBean, split, subAlias ) ;
                    }
                    //----------------------
                    subBean = ( YBean )templateBean.getFieldValue( s ) ;
                    if( subBean != null )
                    {
                        String subAlias = "dao_" + s + "_" ;
                        setOmniValues( disjunction, subBean, split, subAlias ) ;
                    }
                }
            }
        }

        
        String criteriaProjection = beanSend.getCriteriaProjection( key ) ;

        //You can specify what to return, id is default, object is not returned for performance,u can still
        //load all of these objects batch ,never load objects from a query, 100000 results not rare!
        if( !YClientUtils.isBlankTrim( criteriaProjection ) )
        {
            criteria.setProjection( Projections.projectionList( ).add( Projections.property( criteriaProjection ), "id" ) ) ;
        }
        else
        {
            criteria.setProjection( Projections.projectionList( ).add( Projections.property( "id" ), "id" ) ) ;
        }

        //criteria.addOrder( Order.asc( "id" ) ) ;

        if( YClientUtils.isBlankTrim( sort ) )
        {
            criteria.addOrder( Order.desc( "id" ) ) ;
        }
        else
        {
            String realSortPart = sort ;

            if( sort.indexOf( "." ) > -1 )
            {
                String[ ] chainBeans = realSortPart.split( "\\." ) ;

                //If sort path not present in query create it,
                //Example you wanted only users with name John ,
                // (new User()).setName("John")
                //But want to sort by organization name
                // sort = user.deparment.organization
                // so this path must be created if not already exists
                
                for( int z = 0; z < chainBeans.length - 1; z++ )
                {
                    String subAlias = "dao_" + chainBeans[ z ] + "_" ;

                    if( aliasSet.get( subAlias ) != null )
                    {
                        continue ;
                    }

                    Criteria parent = z == 0 ? criteria : aliasSet.get( "dao_" + chainBeans[ z - 1 ] + "_" ) ;

                    Criteria lastCriteria = parent.createCriteria( chainBeans[ z ], subAlias, CriteriaSpecification.LEFT_JOIN ) ;

                    aliasSet.put( subAlias, lastCriteria ) ;

                }

                String subAlias = "dao_" + chainBeans[ chainBeans.length - 2 ] + "_" ;

                if( SortOrder.ASCENDING.equals( order ) )
                {
                    criteria.addOrder( Order.asc( subAlias + "." + chainBeans[ chainBeans.length - 1 ] ) ) ;
                }
                else
                {
                    criteria.addOrder( Order.desc( subAlias + "." + chainBeans[ chainBeans.length - 1 ] ) ) ;
                }
            }
            else
            {
                if( SortOrder.ASCENDING.equals( order ) )
                {
                    criteria.addOrder( Order.asc( sort ) ) ;
                }
                else
                {
                    criteria.addOrder( Order.desc( sort ) ) ;
                }

            }

        }

        if( firstResult != null )
        {
            criteria.setFirstResult( firstResult ) ;
        }

        if( maxResult != null )
        {
            criteria.setMaxResults( maxResult ) ;
        }

        criteria.setCacheable( true ) ;

        List<Integer> results = criteria.list( ) ;

        return results ;

    }
    
    protected static void addExactValues( Criteria criteria, YBean subBean, String alias )
    {
        if( subBean.getId( ) != null && subBean.getId( ).intValue( ) > 0 )
        {
            criteria.add( Restrictions.eq( ( alias != null ? alias + "." : "" ) + "id", subBean.getId( ) ) ) ;
            return ;
        }

        String[ ] fieldNames = subBean.getFieldNames( ) ;

        for( String fieldName : fieldNames )
        {
            String simpleTypeName = ReflectionUtils.getField( subBean.getClass( ), fieldName ).getType( ).getSimpleName( ) ;

            if( "String".equals( simpleTypeName ) )
            {
                if( subBean.getFieldValue( fieldName ) != null )
                {
                    String val = ( String )subBean.getFieldValue( fieldName ) ;

                    if( subBean.query_isForceIncludedField( fieldName ) || ( ( !YClientUtils.isBlankTrim( val ) && !IGNORE_IN_SEARCH.equals( val ) ) && !subBean.query_isForceExcludedField( fieldName ) ) )
                    {
                        criteria.add( Restrictions.like( ( alias != null ? alias + "." : "" ) + fieldName, val ) ) ;
                    }
                }
            }

            if( "int".equals( simpleTypeName ) || "Integer".equals( simpleTypeName ) )
            {
                if( subBean.getFieldValue( fieldName ) != null )
                {
                    Integer ival = ( ( Integer )subBean.getFieldValue( fieldName ) ) ;
                    if( subBean.query_isForceIncludedField( fieldName ) || ( ival != null && ival.intValue( ) > 0 && !subBean.query_isForceExcludedField( fieldName ) ) )
                    {
                        criteria.add( Restrictions.eq( ( alias != null ? alias + "." : "" ) + fieldName, subBean.getFieldValue( fieldName ) ) ) ;
                    }
                }
            }
        }

    } ..............................
    //query_isForceIncludedField  means , we are checking if a value
    protected static void setOmniValues( Disjunction disjunction, YBean subBean, String omni, String alias )
    {
        //This is for sample (unpersistent) beans 
        // Say u created a sample like
        // (new User()).setOrganization( Yorganization.getById(1))
        // you want all users with organization 1 but since organization is attached to session
        //All of its non-lazy values are readen. we are dealing with those ones so omit them
        
        if( subBean.getId( ) != null && subBean.getId( ).intValue( ) > 0 )
        {
            return ;
        }

        String[ ] fieldNames = subBean.getFieldNames( ) ;

        for( String fieldName : fieldNames )
        {
            String simpleTypeName = ReflectionUtils.getField( subBean.getClass( ), fieldName ).getType( ).getSimpleName( ) ;

            if( "String".equals( simpleTypeName ) )
            {
                Object val = subBean.getFieldValue( fieldName ) ;

                if( subBean.query_isForceIncludedField( fieldName ) || ( ( val == null && !IGNORE_IN_SEARCH.equals( val ) ) && !subBean.query_isForceExcludedField( fieldName ) ) )
                {
                    disjunction.add( Restrictions.like( ( alias != null ? alias + "." : "" ) + fieldName, omni ) ) ;
                }
            }
        }

    } 

Log Hibernate event in Database

Sometimes we need to know how many objects created in one session.
(We have very big system. When an order is created there are hundred of objects
created).We need to know all the objects created in one session.


If you are using interceptor you can put a call to logging on those methods.

Below code generates identifiers like this.

H:1967664672,T:43TH:http-bio-80-exec-7

H:1967664672
From session hashcode we can understand which objects created at same hibernate session.

T:43TH:http-bio-80-exec-7
T : Thread Number
H : Thread Name

with thread data, we can understand utilization of threads by server.

How many server threads are running at an interval?
Which users took which thread?

protected static void doEventLog( boolean flag, String event, Session session )
    {
        if( flag )
        {
            YEventLog elog = YEventLog.newInstanceWithStackTrace( event ) ;

            String sessIdent = YClientUtils.truncateStringNoElipsis( "H:" + session.hashCode( ) + ",T:" + Thread.currentThread( ).getId( ) + ";TH:" + Thread.currentThread( ).getName( ), 64 ) ;

            elog.setSessionIdentifier( sessIdent ) ;

            YDelayedPersisterService.delayedSave( elog ) ;
        }
    }
    
    //Put stacktrace of current Action
    public static YEventLog newInstanceWithStackTrace( String eventName )
    {
        YEventLog elog = newInstance( eventName ) ;

        String trace = ExceptionUtils.stackTraceToString( null ) ;

        if( trace.length( ) > 25000 )
        {
            trace = trace.substring( 0, 25000 ) ;
        }

        elog.setStringData( trace ) ;

        return elog ;
    }

public class EventLog
{
    private Integer id ;

    private Integer organization ;

    private Integer user ;

    private String sessionIdentifier ;

    private String eventName ;

    private Date time ;

    private BigDecimal decimalData ;

    private String stringData ;

    private int YVersion ;

    private Date YCreateDate ;

    private Date YUpdateDate ;

}

I suggest you, to have 2 fields in database for holding The session id that object was created
and concatenated ids that object was updated or one concatenate all of these. (1st creator)
A better way will be having a separate table.(Our data is separated over tables)
Session Id, Bean Name, Date, Operation

Korean Food

Korean Food is amazing with its diversity. But that diversity
probably stems from historical famines and Korea landscape.
Korean is like small lowlands between lots of mountains.
So there is not enough place for agriculture for a big population.

Historical droughts lead to scarcity in food. Once a Korean told me
it was not even possible to feed cow or pig ,because they do not have enough food
for them also.
So probably(I wish I could read and investigate much on this topic) they had to try
everything they can. (Worm,dog, whatever Western people laugh)

They made pickles of everything and they are perfect. Especially I love ones
with sesame leaf(perilla). I ate different versions of it. I do not know names,
because it is very hard to memorize anything in Korean. But taste is unforgettable.

Every year I go to Busan for a reason coincidentally(Long story).
Once I discovered a restaurant in Busan. I just want to write it in internet
so that google crawler finds it in searches like below.

Best restaurant in Korea

I do not go to 3 star Michelin ones or real luxury ones, but this place seems so elegant
to me. Price is about 10$-30$. Side dishes are unbelievably perfect. It is in Lotte at near Nampo station.

Korea

100,032 square kilometres

Land use:
arable land: 15.3%
permanent crops: 2.2%
permanent pasture: 0.6%
forest: 63.9%
other: 18.0% (2011)
Irrigated land: 8,804 km² (2003)

Security By Encryption & Programatic Security

How can we add more security when we are writing a security program?( If we are not expert :) )

Think that your rival(attacker,intruder) is able to decrypt RSA or AES in an acceptable time.
Then we have to think on our flown,algorithms.
There are lots of things one can do. I will state some of them.

1) Adding garbage data
Poisson network, disk and database with random garbage data.

2)Partition files
Files are usually encrypted Symmetric. (Asymmetric is so slow for file encryption)
So at the last stage of flow, attacker is left with Symmetric decryption.

What we can do is partition files and encrypt each with Symmetric to
spend more time on file decryption.

3)File entropy
File entropy is randomness in a file.
This is useful in predicting patterns in a file.
One practice about this :

Steganography : concealing file in another file.

We were keeping chunked files in server which were all parts of o file.
We encrypted random parts with Asymmetric .
We made uniform entropy chunks .

*Define a target entropy.
*Calculate file entropy
*Add necessary (distribute over file) data to balance of each chunk.

Cv

Download Cv

ExecutorService Sample

For Thread Management one can use simple Executor Service. We had lots of small job but time consuming jobs which
could execute later. Below is the sample code.

If you want the jobs to be executed in the order you give , use sequential.
Example : You want to fire a register event and then an email notification.
You want these to be executed in the order you submitted to queue.

If you are not interested in the order just use pooled.

protected static ThreadLocal<reentrantlock> requestLock = new ThreadLocal<reentrantlock>( ) ;  
   protected static ThreadLocal<condition> requestCondition = new ThreadLocal<condition>( ) ;  
   public static ReentrantLock getLock( boolean isInit )  
   {  
     ReentrantLock lock = requestLock.get( ) ;  
     if( lock == null &amp;&amp; isInit )  
     {  
       lock = new ReentrantLock( true ) ;  
       requestLock.set( lock ) ;  
     }  
     return lock ;  
   }  
   public static Condition getCondition( boolean isInit )  
   {  
     Condition con = requestCondition.get( ) ;  
     if( con == null &amp;&amp; isInit )  
     {  
       con = getLock( true ).newCondition( ) ;  
       requestCondition.set( con ) ;  
     }  
     return con ;  
   }  
 protected static ExecutorService _sequentialExecutor = null ;  
   protected static ExecutorService _pooledExecutor = null ;  
   static  
   {  
     init( true ) ;  
   }  
   public static void init( boolean isForce )  
   {  
     if( _sequentialExecutor != null &amp;&amp; _pooledExecutor != null &amp;&amp; !isForce )  
     {  
       return ;  
     }  
     _sequentialExecutor = Executors.newSingleThreadExecutor( ) ;  
     _pooledExecutor = Executors.newFixedThreadPool( 4 ) ;  
   }  
   public static <t> Future<t> sequentialExec( YTask<t> task, boolean isWaitForRequest )  
   {  
     if( isWaitForRequest )  
     {  
       ReentrantLock lock = getLock( true ) ;  
       // initialize the condition as well  
       getCondition( true ) ;  
       task.setLock( lock ) ;  
       if( !lock.isHeldByCurrentThread( ) )  
       {  
         lock.lock( ) ;  
       }  
     }  
     return _sequentialExecutor.submit( ( Callable )task ) ;  
   }  
   public static <t> Future<t> sequentialExec( YScheduledTask task )  
   {  
     return _sequentialExecutor.submit( ( Callable )task ) ;  
   }  
   public static <t> Future<t> sequentialExec( Callable<t> callable )  
   {  
     return _sequentialExecutor.submit( callable ) ;  
   }  
   public static <t> Future<t> pooledExec( YTask<t> task, boolean isWaitForRequest )  
   {  
     if( isWaitForRequest )  
     {  
       ReentrantLock lock = getLock( true ) ;  
       // initialize the condition as well  
       getCondition( true ) ;  
       task.setLock( lock ) ;  
       if( !lock.isHeldByCurrentThread( ) )  
       {  
         lock.lock( ) ;  
       }  
     }  
     return _pooledExecutor.submit( ( Callable )task ) ;  
   }  

YTask is a utility class for callable services via this class.

import java.util.concurrent.Callable ;  
 import java.util.concurrent.TimeUnit ;  
 import java.util.concurrent.locks.Condition ;  
 import java.util.concurrent.locks.ReentrantLock ;  
 public class YTask<t> implements Callable<t>, Runnable  
 {  
   protected Callable<t> callable ;  
   protected long delayMs = 0 ;  
   protected ReentrantLock lock = null ;  
   protected Condition condition = null ;  
   public YTask( Callable<t> callable )  
   {  
     this.callable = callable ;  
   }  
   public Callable<t> getCallable( )  
   {  
     return this.callable ;  
   }  
   public long getDelayMs( )  
   {  
     return this.delayMs ;  
   }  
   public ReentrantLock getLock( )  
   {  
     return this.lock ;  
   }  
   public void setLock( ReentrantLock lock )  
   {  
     this.lock = lock ;  
   }  
   public Condition getCondition( )  
   {  
     return this.condition ;  
   }  
   public void setCondition( Condition condition )  
   {  
     this.condition = condition ;  
   }  
   @Override  
   public void run( )  
   {  
     try  
     {  
       call( ) ;  
     }  
     catch( Exception e )  
     {  
       e.printStackTrace( ) ;  
       throw new RuntimeException( "Error during task...", e ) ;  
     }  
   }  
   @Override  
   public T call( ) throws Exception  
   {  
     T res = null ;  
     if( this.delayMs > 0 )  
     {  
       try  
       {  
         Thread.sleep( this.delayMs ) ;  
       }  
       catch( Exception e )  
       {  
         e.printStackTrace( ) ;  
       }  
     }  
     if( this.lock != null )  
     {  
       if( !this.lock.isHeldByCurrentThread( ) )  
       {  
         this.lock.lock( ) ;  
       }  
       System.err.println( "LOCK Info ytask : isHeldByCurrentThread() : " + this.lock.isHeldByCurrentThread( ) + ", getHoldCount() : " + this.lock.getHoldCount( ) + " , " + this.lock.getQueueLength( ) ) ;  
     }  
     try  
     {  
       if( this.callable != null )  
       {  
         if( this.condition != null )  
         {  
           if( !this.condition.await( 180, TimeUnit.SECONDS ) )  
           {  
             sendErrorMessage( null, "Error during waiting for lock in YTask", "Error during waiting for lock in YTask" ) ;  
           }  
         }  
         res = this.callable.call( ) ;  
       }  
       return res ;  
     }  
     finally  
     {  
       if( this.lock != null )  
       {  
         this.lock.unlock( ) ;  
       }  
     }  
   }  
 }  

Running a local http server for dumping output to webpage

Running a local http server for dumping output to webpage.

At our project for encryption ,we had the constraint that we will not be
putting descrypted or open data to network.
There is a client program (Swing) running encrypt decrypt jobs.
Encrypted files are stored at server.

Also users will be able to download encrypted files into their local from a web page.
How could we do that?

1)We generated links like :
http://127.0.0.1:58997/download/-1584127920xe48f773bd522664b1f94d7c926e144bb

2)We opened an Httpserver listening the port specified.

3)User was seeing link in web page.But link was triggering our client program.

4)Program was downloading file parts from server,decrypting them
and writing to response of http request.

Below is piece of codes for this logic.It is not complete but enough.

import java.io.IOException ;
import java.net.InetSocketAddress ;
import java.util.HashMap ;
import java.util.Map ;

import com.sun.net.httpserver.HttpHandler ;
import com.sun.net.httpserver.HttpServer ;

public class KEmbeddedHttpServer 
{
    protected HttpServer server ;

    protected int port ;

    protected Map handlerMap = new HashMap<>( ) ;

    public KEmbeddedHttpServer( int port )
    {
        this.port = port ;
    }

    public HttpServer getServer( )
    {
        return this.server ;
    }

    public int getPort( )
    {
        return this.port ;
    }

    public void setPort( int port )
    {
        this.port = port ;
    }

    public void addHandler( String context, HttpHandler handler )
    {
        this.handlerMap.put( context, handler ) ;
    }

    public void start( ) throws IOException
    {
        this.server = HttpServer.create( new InetSocketAddress( this.port ), 0 ) ;

        for( String context : this.handlerMap.keySet( ) )
        {
            this.server.createContext( "/" + context, this.handlerMap.get( context ) ) ;
        }

        this.server.setExecutor( null ) ; // creates a default executor

        this.server.start( ) ;
    }
}



public void doHandleRequest( final HttpExchange httpExchange ) throws IOException
    {
        String requestPath = httpExchange.getRequestURI( ).getPath( ) ;
        
            try
            {
                

                final String fileName = decFileMeta.substring( 0, decFileMeta.indexOf( ", @" ) ).trim( ) ;

                final OutputStream os = httpExchange.getResponseBody( ) ;

                IKObserver observer = new IKObserver( )
                {
                    @Override
                    public Boolean onAction( String action, byte[ ] data )
                    {
                        try
                        {
                            if( BLOCK_READ.equals( action ) )
                            {
                                os.write( data ) ;
                            }
                            else if( START_OF_OPERATION.equals( action ) )
                            {
                                httpExchange.getResponseHeaders( ).add( "Content-Disposition", "attachment; filename=" + fileName ) ;
                                httpExchange.getResponseHeaders( ).add( "Transfer-encoding", "chunked" ) ;
                                httpExchange.sendResponseHeaders( 200, 0 ) ;
                            }
                            else if( END_OF_OPERATION.equals( action ) )
                            {
                                if( os != null )
                                {
                                    os.flush( ) ;
                                    os.close( ) ;
                                }
                            }

                            return Boolean.TRUE ;
                        }
                        catch( Exception ex )
                        {
                            ex.printStackTrace( ) ;
                            return false ;
                        }
                    }
                } ;


                
            }
            catch( Exception e )
            {
                e.printStackTrace( ) ;
                e.printStackTrace( new PrintWriter( this.writer ) ) ;
                httpExchange.getResponseHeaders( ).add( "Content-Disposition", "inline; filename=error.html" ) ;
                httpExchange.sendResponseHeaders( 500, this.baos.size( ) ) ;
            }

        

    }


 

How to write a zero knowledge server for encryption?

In our project for encryption we needed to make server as blind as possible.

We applied below algorithms for this purpose.

1) Server must not know which files it is storing.
Put files in parts.
Place them at random locations on server.
Put fake blocks.

2)Server file parts must be meaningless.
For attacking for basic files(raw text files) one can check occurring patterns for frequency analysis.
So zip files to decrease entropy.

3)While checking database attackers must not understand anything.
For example think you have a column as user id.

If attacker sees column id = 3 he will understand all files of user 3.
So every column must be kept encrypted.
But a hash or md5 of 3 will always be same .Ex : 344jkfjkdf994jjfjf
So an attacker can understand these files belong to same user.

So what is a good candidate for every row to have different values for same data?
Row id . Put row id into hashing or AES algorithm .

But still there is a pattern.
User Id + Row id
either post or prefix can give a clue.

So put time stamp at beginning.

Date.now + user id + row id

Every user must understand his files but not other people' files.
So AES this with user AES key


select CAST( AES_DECRYPT( UNHEX( enc_column), ? ) as char ) as user_info

FROM anytable_encyrpted HAVING user_info = concat( 'userid=(', ?, '),id=(', anytable_encyrpted_id , ')' ) " ;

? are parameters as follows :
User AES key, and user id

How to simulate a thumb driver in local

In a project we needed to simulate a thumb driver for multi user system ,
share public keys over network.


A thumb driver does not share it's private key.
We can simulate a thumb driver in our local.


Project will enable users of system to use their local laptops or
usb drives as thumb drive.

So let's write basic principles for this simulation.
1)All encryption decryption must occur at local machine.

2)All data send over network must be encrypted

3)User public keys must be kept at server.(Public key distribution server better to
be on a different server than data server)

4)RSA is slow for file encryption.Encrypt Files with AES.
Encrypt AES keys with every user's public key put to server.
***You have one door,one key.
for n users clone key and put in a box only that target user have key(user's private key)

Lucene Tester

There is no need to explain why someone needs Lucene .
There are also lots of samples over net. I am just putting our sample if anyone
encounters this link.

import java.io.File ;
import java.io.Reader ;
import java.io.Serializable ;
import java.io.StringReader ;
import java.util.HashMap ;
import java.util.List ;

import org.apache.lucene.analysis.Analyzer ;
import org.apache.lucene.analysis.standard.StandardAnalyzer ;
import org.apache.lucene.document.Document ;
import org.apache.lucene.document.Field ;
import org.apache.lucene.index.CorruptIndexException ;
import org.apache.lucene.index.IndexReader ;
import org.apache.lucene.index.IndexWriter ;
import org.apache.lucene.index.IndexWriter.MaxFieldLength ;
import org.apache.lucene.index.Term ;
import org.apache.lucene.queryParser.QueryParser ;
import org.apache.lucene.search.IndexSearcher ;
import org.apache.lucene.search.Query ;
import org.apache.lucene.search.ScoreDoc ;
import org.apache.lucene.search.Searcher ;
import org.apache.lucene.search.TermQuery ;
import org.apache.lucene.search.TopDocs ;
import org.apache.lucene.search.similar.MoreLikeThis ;
import org.apache.lucene.search.spell.Dictionary ;
import org.apache.lucene.search.spell.LuceneDictionary ;
import org.apache.lucene.search.spell.SpellChecker ;
import org.apache.lucene.store.Directory ;
import org.apache.lucene.store.FSDirectory ;
import org.apache.lucene.util.Version ;

public class YLuceneTester
{


    private final String indexDir = "D:\\indexDir" ;

    private final String spellDirPath = "D:\\spellDir" ;

    /**
     * create index
     */
    public boolean createIndex( ) throws Exception
    {
        //        if( true == ifIndexExist( ) )
        //        {
        //            return true ;
        //        }
        //        File dir = new File(dataDir);
        //        if(!dir.exists()){
        //            return false;
        //        }

        //File[] htmls = dir.listFiles();

        Directory fsDirectory = FSDirectory.open( new File( this.indexDir ) ) ;
        Analyzer analyzer = new StandardAnalyzer( Version.LUCENE_33 ) ;
        IndexWriter indexWriter = new IndexWriter( fsDirectory, analyzer, true, MaxFieldLength.UNLIMITED ) ;

        addDocument( indexWriter ) ;

        indexWriter.optimize( ) ;
        indexWriter.close( ) ;

        IndexReader indexReader = null ;
        try
        {
            indexReader = IndexReader.open( fsDirectory ) ;
            Dictionary dictionary = new LuceneDictionary( indexReader, "trans" ) ;
            FSDirectory spellDir = FSDirectory.open( new File( this.spellDirPath ) ) ;
            SpellChecker spellChecker = new SpellChecker( spellDir ) ;
            spellChecker.indexDictionary( dictionary ) ;
            spellChecker.close( );
        }
        finally
        {
            if( indexReader != null )
            {
                indexReader.close( ) ;
            }
        }
        return true ;

    }

    /**
     * Add one document to the Lucene index
     * @throws Exception 
     * @throws CorruptIndexException 
     */
    public void addDocumentDB( IndexWriter indexWriter ) throws CorruptIndexException, Exception
    {
        YOrganization org = YOrganization.getTopLevelOrganization( "TXG100" ) ;

        String hql = " Select y.trans,y.id FROM YEntityTranslation y where y.organization.id =  " + org.getId( ) ;

        HashMap parameters = new HashMap( ) ;

        List existingCatalogData = ( List )HibernateUtils.execHQL( hql, parameters, 0, 1000 ) ;          for( Object[ ] datas : existingCatalogData )         {             String trans = ( String )datas[ 0 ] ;             if( YClientUtils.isBlankTrim( trans ) )                 continue ;              System.err.println( trans ) ;             Document document = new Document( ) ;             //document.add( new Field( "path", path, Field.Store.YES, Field.Index.NO ) ) ;             document.add( new Field( "trans", trans, Field.Store.YES, Field.Index.ANALYZED ) ) ;              indexWriter.addDocument( document ) ;         }      }          public void addDocument( IndexWriter indexWriter ) throws CorruptIndexException, Exception     {         String[] items = new String[]{"African lion","African wild cat","African wild dog","dog","cat","lion"};                  for( String item : items )         {              Document document = new Document( ) ;             document.add( new Field( "trans", item, Field.Store.YES, Field.Index.ANALYZED ) ) ;              indexWriter.addDocument( document ) ;         }      }      public Query suggest( String queryString ,int distance) throws Exception     {         try         {             Directory fsDirectory = FSDirectory.open( new File( this.spellDirPath ) ) ;             SpellChecker spellChecker = new SpellChecker( fsDirectory ) ;             if( spellChecker.exist( queryString ) )             {                 return null ;             }             String[ ] similarWords = spellChecker.suggestSimilar( queryString, distance ) ;             if( similarWords.length == 0 )             {                 return null ;             }              System.err.println( " Term = " + queryString + " Suggestions :" ) ;             for( String similarWord : similarWords )             {                 System.err.println( " ) " + similarWord ) ;             }              return new TermQuery( new Term( "trans", similarWords[ 0 ] ) ) ;         }         catch( Exception e )         {             throw new Exception( e.getMessage( ) ) ;         }     }      public void searchIndex( String[ ] queryStrings ) throws Exception     {         Searcher searcher = new IndexSearcher( FSDirectory.open( new File( this.indexDir ) ) ) ;         QueryParser parser = new QueryParser( Version.LUCENE_CURRENT, "trans", new StandardAnalyzer( Version.LUCENE_CURRENT ) ) ;         for( String queryString : queryStrings )         {             System.out.println( "nsearching for: " + queryString ) ;             Query query = parser.parse( queryString ) ;             TopDocs results = searcher.search( query, 10 ) ;             System.out.println( "total hits: " + results.totalHits ) ;             ScoreDoc[ ] hits = results.scoreDocs ;             for( ScoreDoc hit : hits )             {                 Document doc = searcher.doc( hit.doc ) ;                 System.out.printf( "%5.3f %sn \n", hit.score, doc.get( "trans" ) ) ;             }         }         searcher.close( ) ;     }       /**      * judge if the index exists already      */     public boolean ifIndexExist( )     {         File directory = new File( this.indexDir ) ;         if( 0 < directory.listFiles( ).length )
        {
            return true ;
        }
        else
        {
            return false ;
        }
    }

    public String getIndexDir( )
    {
        return this.indexDir ;
    }


    public Query parse( String queryString ) throws Exception
    {
        QueryParser queryParser = new QueryParser( Version.LUCENE_CURRENT, "trans", new StandardAnalyzer( Version.LUCENE_CURRENT ) ) ;
        queryParser.setDefaultOperator( QueryParser.AND_OPERATOR ) ;
        return queryParser.parse( queryString ) ;
    }

    public void search( String queryString ,int distance ) throws Exception
    {
        long startTime = System.currentTimeMillis( ) ;
        IndexSearcher is = null ;
        FSDirectory spellDir = FSDirectory.open( new File( this.spellDirPath ) ) ;
        Directory fsDirectory = FSDirectory.open( new File( this.indexDir ) ) ;

        int minimumHits = 100 ;
        int minimumScore = 5 ;

        try
        {
            is = new IndexSearcher( fsDirectory ) ;
            Query query = parse( queryString ) ;

            TopDocs tdocs = is.search( query, 100 ) ;

            //Hits hits = is.search( query ) ;

            //            for( ScoreDoc sdoc :  tdocs.scoreDocs )
            //            {
            //                sdoc.
            //            }

            String suggestedQueryString = null ;
            if( tdocs.totalHits < minimumHits || tdocs.getMaxScore( ) < minimumScore )
            {
                Query didYouMean = suggest( queryString ,distance) ;
                if( didYouMean != null )
                {
                    suggestedQueryString = didYouMean.toString( "trans" ) ;
                }
            }

            long endTime = System.currentTimeMillis( ) ;

            //return new SearchResult( extractHits( hits ), hits.length( ), endTime - startTime, queryString, suggestedQueryString ) ;
        }
        finally
        {
            if( is != null )
            {
                is.close( ) ;
            }
        }
    }

    public void moreLikeThis( String text ) throws Exception
    {
        Directory fsDirectory = FSDirectory.open( new File( this.indexDir ) ) ;

        IndexReader indexReader = IndexReader.open( fsDirectory ) ;

        //        FuzzyLikeThisQuer flt = new FuzzyLikeThisQuery( 50, new StandardAnalyzer( ) ) ;
        //        flt.addTerms( "product critical update", "title", 0.75f, FuzzyQuery.defaultPrefixLength ) ;
        //        BooleanQuery q = ( BooleanQuery )flt.rewrite( r ) ;
        //        int minNumClauseMatches = Math.round( q.clauses( ).size( ) * 0.5f ) ;
        //        q.setMinimumNumberShouldMatch( minNumClauseMatches ) ;

        IndexSearcher is = new IndexSearcher( FSDirectory.open( new File( this.indexDir ) ) ) ;

        MoreLikeThis mlt = new MoreLikeThis( indexReader ) ;
        mlt.setFieldNames( new String[ ] { "trans" } ) ;

        mlt.setMinWordLen( 2 ) ;
        mlt.setBoost( true ) ;

        Reader reader = new StringReader( text ) ;

        //Create the query that we can then use to search the index
        Query query = mlt.like( reader ) ;

        //Search the index using the query and get the top 5 results
        TopDocs topDocs = is.search( query, 5 ) ;

        //Create an array to hold the quotes we are going to
        //pass back to the client

        for( ScoreDoc scoreDoc : topDocs.scoreDocs )
        {
            //This retrieves the actual Document from the index using
            //the document number. (scoreDoc.doc is an int that is the

            System.err.print( "--" + scoreDoc.toString( ) ) ;
        }
        
        is.close( );

    }
    
    public static void init()
    {
        YLuceneTester luceneTester = new YLuceneTester( ) ;
        try
        {
            luceneTester.createIndex( ) ;

            //luceneTester.searchIndex( new String[ ] { "Cleaner" } ) ;
            //TermQuery q = ( TermQuery )luceneTester.suggest( "Claner" ) ;
            //q.extractTerms( terms )

            //luceneTester.moreLikeThis( "Clean" ) ;

            //            luceneTester.search( "Cleaner" ) ;
            //            luceneTester.search( "Cordless " ) ;
            //
            //            luceneTester.suggest( "Cleane" ) ;
            //            luceneTester.suggest( "Clean" ) ;
            //            luceneTester.suggest( "Clnr" ) ;

        }
        catch( Exception e )
        {
            e.printStackTrace( ) ;
        }
    }
    
    public static void tests()
    {
        YLuceneTester luceneTester = new YLuceneTester( ) ;
        try
        {

//            luceneTester.searchIndex( new String[ ] { "Afri" } ) ;
//            luceneTester.searchIndex( new String[ ] { "African" } ) ;
//            luceneTester.searchIndex( new String[ ] { "Africax" } ) ;
//            TermQuery q = ( TermQuery )luceneTester.suggest( "Claner" ) ;
//            q.extractTerms( terms )

            //luceneTester.moreLikeThis( "dog" ) ;

                        luceneTester.search( "Afrieen" ,1 ) ;
                        luceneTester.search( "Afrieen" ,1 ) ;
                        luceneTester.search( "Afrieen" ,1 ) ;
            //            luceneTester.search( "Cordless " ) ;
            //
                        luceneTester.suggest( "Afrieen",2 ) ;
                        luceneTester.suggest( "lion" ,2) ;
            //            luceneTester.suggest( "Clnr" ) ;

        }
        catch( Exception e )
        {
            e.printStackTrace( ) ;
        }
    }

    public static void main( String[ ] args )
    {
        //HibernateUtils._configFileName = "hibernate.hqltest.xml" ;
        
        //init( );
        
        tests( );



    }
}

Display Current Sequence of All tables In xhtml

At a project Postgre sequences were cached and sometimes at server restarts we were seeing
strange behaviours. Also when someone inserts a sequence manually we were again having difficulties.
Here is simple page to see your database state.

public ArrayList>  initDbStateList(List            tableNames ,HashMap tableNamePrimary )
   {
	   ArrayList> data = new ArrayList>();
	   
	   
      
      for( String tname : tableNames )
      {
         HashMap map = new HashMap();
         
         map.put("tname", tname);
         map.put("seqid", 0L);
         map.put("maxid", 0L);
         
         try {
            
         
         
         String sql = "select nextval('MYDB."+ tname+"_sequence"+"');"; 
         Long seqid = -1L; 
                  
         try {
            seqid = queryForLong(sql);
         }
         catch (Exception e) {
            logger.error(e, e);
         }
          
         
         map.put("seqid", seqid);
         
         String maxsql = " select max("+tableNamePrimary.get(tname)+") from nufus."+tname+"; ";          
         Long maxid = queryForLong(maxsql);
         map.put("maxid", maxid);
         
         if( seqid != null && maxid != null )
         {
            if( seqid < maxid)
            {
               map.put("problematic", "YES");
            }
         }
         
         
         data.add(map);
         }
         catch (Exception e) {
            map.put("problematic", "NONEXISTENT");
            e.printStackTrace();
         }
      
      
	   }
     
	   
      return data;
   }

 

And datatable to display the result will be like :

JSF Style Verifier

In a JSF project we had a very long list of principles for UI components.

Example : p:panel inside p:fieldset must have style ed_borderless

I was told to check screens for these things. Unfortunately I am not good at
design things. Most people were able to say what is wrong in a panel at 1st glance.
But I was not.

I taught i can write a verifier for style and process all of pages at once.

One more thing about project was ,it's GUI was so heavy so there was deep
path for components.

p:panelGrid : ->p:column : ->p:row : ->p:panelGrid :

I put a search depth parameter to check for under each component.


Program will geneate an output like below :


p:panel inside h:form must have reset_uipanel : missing style (reset_uipanel) having ed_borderless
>>> h:form : mytForm->p:dialog : personDialog->h:form : form->ui:define : ->ui:composition : ->h:body : ->f:view : ->html : ->

It tells which style is missing and missing path.

import java.io.File ;
import java.util.ArrayList ;
import java.util.HashMap ;
import java.util.HashSet ;

import javax.xml.parsers.DocumentBuilder ;
import javax.xml.parsers.DocumentBuilderFactory ;

import org.w3c.dom.Document ;
import org.w3c.dom.Element ;
import org.w3c.dom.NamedNodeMap ;
import org.w3c.dom.Node ;
import org.w3c.dom.NodeList ;



public class StyleVerifier
{

    public static String NODE_Panel = "p:panel" ;

    public static String NODE_Panelgrid = "p:panelgrid" ;

    protected StringBuffer logs = new StringBuffer( ) ;

    public int globalCheckDepth = -1 ;

    public StringBuffer getLogs( )
    {
        return this.logs ;
    }

    public void setLogs( StringBuffer logs )
    {
        this.logs = logs ;
    }

    public void getChildsOfType( Node node, String type, ArrayList res )
    {

        if( res == null )
        {
            res = new ArrayList( ) ;
        }

        NodeList nodeList = node.getChildNodes( ) ;

        for( int i = 0; i < nodeList.getLength( ); i++ )
        {

            Node n = nodeList.item( i ) ;

            System.err.println( " verifying " + n.getNodeName( ) ) ;

            if( n.getNodeName( ).equals( type ) )
            {

                Node actualNode = n.getFirstChild( ) ;

                if( actualNode != null )
                {
                    res.add( actualNode ) ;
                }
            }

            getChildsOfType( n, type, res ) ;
        }

    }

    public String getParentPath( Element element )
    {

        if( !( element.getParentNode( ) instanceof Element ) )
            return "" ;

        Element par = ( Element )element.getParentNode( ) ;

        if( par == null )
            return "" ;

        String nodDisp = par.getNodeName( ) + " : " + par.getAttribute( "id" ) ;

        return nodDisp + "->" + getParentPath( par ) ;
    }

    //breakingParent  is for items like panel.
    //If panel is inside h:form before fieldset we can not say it is under fieldset.
    public Element hasParentOfType( Element element, String parentType, String breakingParent, int checkDepth )
    {
        if( checkDepth == 0 )
            return null ;

        if( !( element.getParentNode( ) instanceof Element ) )
            return null ;

        Element par = ( Element )element.getParentNode( ) ;

        if( par == null )
            return null ;

        String nodnam = par.getNodeName( ) ;

        if( breakingParent != null && breakingParent.equals( nodnam ) )
        {
            return null ;
        }

        if( parentType.equals( nodnam ) )
        {
            return par ;
        }

        checkDepth-- ;
        return hasParentOfType( par, parentType, breakingParent, checkDepth ) ;
    }

    public boolean isPanel( Element element )
    {
        String nodnam = element.getNodeName( ) ;

        System.out.println( nodnam + "=" + element.getAttribute( "styleClass" ) ) ;

        if( NODE_Panel.equalsIgnoreCase( nodnam ) )
        {
            return true ;
        }
        return false ;
    }

    public boolean isType( Element element, String type )
    {
        String nodnam = element.getNodeName( ) ;

        if( type.equalsIgnoreCase( nodnam ) )
        {
            return true ;
        }
        return false ;
    }

    public void validateFile( File fileToVerify ) throws Exception
    {

        if( !fileToVerify.getName( ).endsWith( ".xhtml" ) )
            return ;

        System.err.println( "Parse beginnig" ) ;
        
        //"D:\verify\webapp\"
        

        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance( ) ;
        dbf.setNamespaceAware( false ) ;
        dbf.setValidating( false ) ;
        dbf.setFeature( "http://xml.org/sax/features/namespaces", false ) ;
        dbf.setFeature( "http://xml.org/sax/features/validation", false ) ;
        dbf.setFeature( "http://apache.org/xml/features/nonvalidating/load-dtd-grammar", false ) ;
        dbf.setFeature( "http://apache.org/xml/features/nonvalidating/load-external-dtd", false ) ;

        DocumentBuilder db = dbf.newDocumentBuilder( ) ;
        Document document = db.parse( fileToVerify ) ;

        NodeList alllist = document.getElementsByTagName( "*" ) ;

        System.err.println( "Parse step 2" ) ;

        for( int i = 0; i < alllist.getLength( ); i++ )
        {

            Element element = ( Element )alllist.item( i ) ;

            verifyStyleUnderParent( element, "p:panel", "h:form", "p:fieldset", "reset_uipanel", "p:panel inside h:form must have reset_uipanel", this.globalCheckDepth ) ;

            System.err.println( "Parse step 3" ) ;

            verifyStyleUnderParent( element, "p:panel", "p:fieldset", "h:form", "ed_borderless", "p:panel inside p:fieldset must have ed_borderless", this.globalCheckDepth ) ;

            System.err.println( "Parse step 4" ) ;

            verifyStyleUnderParent( element, "p:panelgrid", "p:panelgrid", null, "ed_gridcellborderless", "p:panelgrid inside p:panelgrid must have ed_gridcellborderless", this.globalCheckDepth ) ;

            System.err.println( "Parse step 5" ) ;

            //
            HashMap styleValues = new HashMap( ) ;
            styleValues.put( "closable", "false" ) ;

            HashSet notNullStyles = new HashSet( ) ;
            notNullStyles.add( "header" ) ;

            verifyStyleMustHave( element, "p:dialog", styleValues, notNullStyles, "Dialog not having proper rules" ) ;
        }

    }




    public String verifyStyleUnderParent( Element element, String mytype, String parenttype, String breakingParent, String mustHaveStyle, String breakMessage, int checkDepth )
    {

        boolean isTargetType = isType( element, mytype ) ;
        if( isTargetType )
        {
            boolean isInside = hasParentOfType( element, parenttype, breakingParent, checkDepth ) != null ;

            if( isInside )
            {

                String styleClass = element.getAttribute( "styleClass" ) ;
                if( styleClass == null || !mustHaveStyle.equals( styleClass ) )
                {

                    String msg = element.getAttribute( "id" ) + " missing style (" + mustHaveStyle + ") having " + styleClass + "\n" ;
                    msg += ">>> " + getParentPath( element ) + "\n\n" ;

                    this.logs.append( breakMessage + " : " + msg ) ;
                    this.logs.append( getItemXML( element ) ) ;
                    
                    return msg ;
                }
            }
        }

        return null ;
    }

    
    public String getItemXML( Element element )
    {
        NamedNodeMap nameNodeMap = element.getAttributes( ) ;

        String s = "<" + element.getNodeName( ) + " " ;
        for( int i = 0; i < nameNodeMap.getLength( ); ++i )
        {
            Node attr = nameNodeMap.item( i ) ;
            s += " " + attr.getNodeName( ) + "=\"" + attr.getNodeValue( ) + "\"" ;
        }
        return s + " >\n\n" ;

    }

    //Checks whether element have stated styles with stated values
    public String verifyStyleMustHave( Element element, String mytype, HashMap styleValues, HashSet notNullStyles, String breakMessage ) throws Exception
    {

        String missingStyles = "" ;
        boolean isTargetType = isType( element, mytype ) ;
        if( isTargetType )
        {
            for( String key : styleValues.keySet( ) )
            {
                String styleClass = element.getAttribute( key ) ;
                String mustHaveStyle = styleValues.get( key ) ;
                if( styleClass == null || !mustHaveStyle.equals( styleClass ) )
                {

                    String msg = element.getAttribute( "id" ) + " , " + key + " must style (" + mustHaveStyle + ") having " + styleClass + "\n" ;
                    msg += ">>> " + getParentPath( element ) + "\n\n" ;

                    missingStyles += msg ;
                }
            }

            for( String key : notNullStyles )
            {
                String styleClass = element.getAttribute( key ) ;
                if( styleClass == null )
                {

                    String msg = "id= " + element.getAttribute( "id" ) + " , " + key + " missing style   \n" ;
                    msg += ">>> " + getParentPath( element ) + "\n\n" ;

                    missingStyles += msg ;
                }
            }
        }

        if( missingStyles != null && missingStyles.trim( ).length( ) > 0 )
        {
            this.logs.append( getItemXML( element ) ) ;
            this.logs.append( breakMessage + " : " + missingStyles ) ;

        }

        return null ;
    }

    //h:form içerisindeki p:panel reset_uipanel
    //fieldset içerisindeki p:panel        ed_borderless
    //p:panelgrid içerisindeki p:panelgrid        ed_gridcellborderless

    public void verifyFileOrDirectory( File fileOrDir ) throws Exception
    {

        if( fileOrDir.isDirectory( ) )
        {
            File[ ] subitems = fileOrDir.listFiles( ) ;

            for( File subitem : subitems )
            {

                this.verifyFileOrDirectory( subitem ) ;
            }

        }
        else
        {
            this.logs.append( fileOrDir.getName( ) + "\n\n" ) ;
            
            String absolutePath = fileOrDir.getAbsolutePath( );
            String plink = "\nlocalhost:7001/aydes" +absolutePath.substring( absolutePath.indexOf( "webapp" ) +"webapp".length( ) );
            plink = plink.replaceAll( "xhtml", "jsf" ) + "\n";
            
            this.logs.append( plink + "\n\n" ) ;
            
            System.err.println( fileOrDir.getAbsolutePath( ) ) ;

            if( !fileOrDir.getName( ).endsWith( ".xhtml" ) )
                return ;

            //validateFile( FileUtils.fileToString( fileOrDir.getAbsolutePath( ), "utf-8" ) );

            validateFile( fileOrDir ) ;

            
        }

    }

    public int getGlobalCheckDepth( )
    {
        return this.globalCheckDepth ;
    }

    public void setGlobalCheckDepth( int globalCheckDepth )
    {
        this.globalCheckDepth = globalCheckDepth ;
    }

    public static void runForDepth( int depth, String path )
    {
        StyleVerifier uiVerifier_ByXml = new StyleVerifier( ) ;
        try
        {
            uiVerifier_ByXml.setGlobalCheckDepth( depth ) ;
            uiVerifier_ByXml.verifyFileOrDirectory( new File( path ) ) ;
            
            FileUtils.stringToFile( "D:\\verify_log_" + uiVerifier_ByXml.globalCheckDepth + ".txt", uiVerifier_ByXml.logs.toString( ), "utf-8", true ) ;
            uiVerifier_ByXml.logs.setLength( 0 ) ;

        }
        catch( Exception e )
        {
            // TODO Auto-generated catch block
            e.printStackTrace( ) ;
        }
    }

    public static void main( String[ ] args )
    {

        String sample = "D:\\verify" ;

        StyleVerifier uiVerifier_ByXml = new StyleVerifier( ) ;
        try
        {
            runForDepth( -1, sample ) ;
            runForDepth( 1, sample ) ;
            runForDepth( 2, sample ) ;
            runForDepth( 3, sample ) ;
            runForDepth( 4, sample ) ;
        }
        catch( Exception e )
        {
            e.printStackTrace( ) ;
        }

        System.err.println( uiVerifier_ByXml.logs.toString( ) ) ;

    }

}

Lazy load exception , Re-using invalid objects

When you get an exception on hibernate session you can not use those objects again.
Their session is invalidated. You must close session and not use objects loaded in that session.
However for some reasons(logging,marking) You need to use them.


What you can do is,load all objects as proxies.
Below code is not complete(reference methods not here) but it is clear and you can easily understand the idea.

    public void refreshBeans( )
    {
        String[ ] beans = this.getBeanNames( ) ;

        for( String beanName : beans )
        {
            YBean val = ( YBean )this.getFieldValue( beanName ) ;

            if( val != null && val.getId( ) != null && val.getId( ).intValue( ) != 0 )
            {
                YBean newbean = HibernateUtils.loadObjectProxyById( ReflectionUtils.getRealBeanClass( val.getClass( ) ), val.getId( ) ) ;
                this.setFieldValue( beanName, newbean ) ;
            }
        }
    }


 

Trace All opened sessions on hibernate

For bench marking issues we needed to trace all opened sessions by hibernate.
We were also tracing when it was closed and which objects were loaded during session.
Idea was simple.


Exception ex = new Exception( "Session opened on " + CalendarUtils.now( ) ) ;


This uniquely identifies the current path of execution and since date is added it is unique.



Example : below code puts a mark a debug object on transaction trace pool. So that we can find
when begined the problematic transaction began.

 public static void beginTransaction( )
    {
        getSession( ).beginTransaction( ) ;

        Exception ex = new Exception( "Transaction began on " + CalendarUtils.now( ) ) ;

        Integer code = getSessionStackTraceIdent( ).get( ) ;

        getTransactionTracePool( ).put( code, ex ) ;

        getTransactionTimestampPool( ).put( code, CalendarUtils.now( ) ) ;

        System.err.println( "Transaction Open in beginTransaction " + getSession( ).getTransaction( ).hashCode( ) ) ;
    }
 

Dump objects in current hibernate session

Sometimes u want to monitor all objects loaded in current hibernate session. For this we get "persistenceContext" from hibernate session. Since it is not visible we played with visibility. Below is a nice very handy piece of code doing this. You can this method at commit of hibernate session to check how many objects loaded during transaction. Most of time you will be very surprised. It will help you to optimize your code.

 public static Object getField( Object instance, String name ) throws Exception
    {
        Field f = instance.getClass( ).getDeclaredField( name ) ;
        f.setAccessible( true ) ;
        return f.get( instance ) ;
    }


public static String printHibernateEntities( )
    {
        final Session s = HibernateManager.getSession( ) ;

        if( s == null )
            return null ;

        StringBuffer sbf = new StringBuffer( ) ;

        List result = null ;
        try
        {
            StatefulPersistenceContext spconContext = ( StatefulPersistenceContext )getField( s, "persistenceContext" ) ;
            Map entities = spconContext.getEntityEntries( ) ;

            result = new ArrayList( entities.size( ) ) ;
            for( Iterator it = entities.values( ).iterator( ); it.hasNext( ); )
            {
                Object o = it.next( ) ;

                String className = ( String )getField( o, "entityName" ) ;

                if( className.indexOf( "." ) > 0 )
                    className = className.substring( className.lastIndexOf( '.' ) + 1, className.length( ) ) ;

                Integer id = ( Integer )getField( o, "id" ) ;

                org.hibernate.engine.EntityEntry e = ( org.hibernate.engine.EntityEntry )o ;

                result.add( className + ": " + id + " = " + o.hashCode( ) + ", key: " + ( e.getId( ) == null ? "N/A" : e.getEntityKey( ) ) + ", status: " + e.getStatus( ) ) ;
            }



            for( int i = 0; i < result.size( ); i++ )
            {

                sbf.append( i + " ) " + result.get( i ) + "\n" ) ;
            }

            if( !FileUtils.isUnix( ) )
            {
                System.out.println( "printHibernateEntities" ) ;
                System.out.println( sbf.toString( ) ) ;
            }
        }
        catch( Exception e )
        {
            e.printStackTrace( ) ;
            YEmergencyEmail.sendErrorMessage( e, "Can not get a dump of printHibernateEntities ", null ) ;
        }

        return sbf.toString( ) ;
    }

illegally attempted to associate a proxy with two open Sessions

We were having this exception in a very complicated process where hundreds of objects were being saved or updated in a single work unit.(No it was natural system was so) If this happens in simple block or hibernate gives any clue it is easy to solve. But sometimes there is no clue so we wrote a DefaultSaveOrUpdateEventListener. As we guess it is because of parentsByChild ( Parent entities cache by their child for cascading ) map at PersistenceContext we wrote a code at bottom. It checks for the item that get exception and which beans are waiting for cascade. This revealed us the problem.

protected static void initSaveOrUpdateEventListenerHook( Configuration config )
    {
        List l = new ArrayList( ) ;
        SaveOrUpdateEventListener[ ] listeners = config.getEventListeners( ).getSaveOrUpdateEventListeners( ) ;
        l.add( new DefaultSaveOrUpdateEventListener( )
        {

            

            @Override
            public void onSaveOrUpdate( org.hibernate.event.SaveOrUpdateEvent arg0 )
            {
                try
                {
                    super.onSaveOrUpdate( arg0 ) ;
                }
                catch( RuntimeException e )
                {
                    e.printStackTrace( ) ;

                    try
                    {
                        Object pc = arg0.getSession( ).getPersistenceContext( ) ;

                        Map m = ( Map )getField( pc, "parentsByChild" ) ;

                        // System.err.println( "Map size: " + m.size( ) + ", " + m ) ;

                        StringBuilder sb = new StringBuilder( ) ;

                        int cnt = 0 ;

                        Map.Entry[] list = IdentityMap.concurrentEntries( m ) ;

                        for( Map.Entry me : list )
                        {
                            YBean childBean = ( YBean )me.getKey( ) ;
                            YBean parentBean = ( YBean )me.getValue( ) ;

                            sb.append( "item " + cnt++ + " is " + parentBean.getClass( ).getSimpleName( ) + "#" + parentBean.getId( ) + ", child is " + childBean.getClass( ).getSimpleName( ) + "#" + childBean.getId( ) + "
\n" ) ;
                        }

                        System.err.println( "the child parent Map is " + sb ) ;

                    }
                    catch( Exception e1 )
                    {
                        e1.printStackTrace( ) ;
                        YEmergencyEmail.sendErrorMessage( e1, "Error on child parent map mailing", null ) ;
                    }

            } ;

            @Override
            protected boolean reassociateIfUninitializedProxy( Object object, SessionImplementor source )
            {
                // System.err.println( "Object is ************* " + object ) ;
                try
                {
                    return super.reassociateIfUninitializedProxy( object, source ) ;
                }
                catch( RuntimeException e )
                {
                    e.printStackTrace( ) ;

                    try
                    {
                        YBean bean = ( YBean )object ;

                        String uid = "UID_" + System.nanoTime( ) ;

                        int hashCode = -1 ;
                        if( bean instanceof HibernateProxy )
                        {
                            HibernateProxy prox = ( HibernateProxy )bean ;

                            hashCode = prox.getHibernateLazyInitializer( ).getSession( ).hashCode( ) ;

                            Exception sessionTrace = HibernateManager.getSessionTracePool( ).get( hashCode ) ;

                            System.err.println( "uid : ========================= session is from =========================== " ) ;
                            sessionTrace.printStackTrace( ) ;

                            YEmergencyEmail.sendErrorMessage( sessionTrace, "Session " + hashCode + "uid 3:" + uid + "session is from", "uid:" + uid + " session is from reassociateIfUninitializedProxy " ) ;
                        }

                    }
                    catch( Exception e2 )
                    {
                        e2.printStackTrace( ) ;
                        YEmergencyEmail.sendErrorMessage( e2, "Error on cachting exception in reassociateIfUninitializedProxy", null ) ;
                    }

                    throw e ;
                }
            }
        } ) ;

        l.addAll( Arrays.asList( listeners ) ) ;
        SaveOrUpdateEventListener[ ] newListeners = l.toArray( new SaveOrUpdateEventListener[ l.size( ) ] ) ;
        config.getEventListeners( ).setSaveOrUpdateEventListeners( newListeners ) ;
    }

Connecting to SFTP

SFTP is a network protocol that provides file operation over network .

It  runs on a secure channel.

Below is a sample code for SFTP actions.

 protected JSch jsch = null ;



    protected Session session = null ;



    protected ChannelSftp sftpChannel = null ;





    /** A convenience method for connecting and logging in */

    public boolean connectAndLogin( String host, String userName, String password, String privateKey, String publicKey ) throws IOException, JSchException

    {

        boolean success = false ;



        try

        {

            System.err.println( "Tryign to SECURELY connect " + host + " with user name " + userName + ", password: ************ " ) ;



            this.jsch = new JSch( ) ;



            // will this work?

            java.security.Security.removeProvider( "SunPKCS11-Solaris" ) ;



            JSch.setLogger( this ) ;



            if( !YClientUtils.isBlankTrim( privateKey ) && !YClientUtils.isBlankTrim( publicKey ) )

            {

                this.jsch.addIdentity( userName, privateKey.getBytes( "utf8" ), publicKey.getBytes( "utf8" ), password == null ? null : password.getBytes( "uft8" ) ) ;

            }

            else

            {

                throw new IllegalArgumentException( "Private and public keys must not be null" ) ;

            }



            this.session = this.jsch.getSession( userName, host ) ;



            this.session.setConfig( "StrictHostKeyChecking", "no" ) ;



            this.session.setConfig( "PreferredAuthentications", "publickey" ) ;

            this.session.setTimeout( 120000 ) ;

            

            this.session.connect( ) ;



            Channel channel = this.session.openChannel( "sftp" ) ;

            channel.connect( ) ;



            this.sftpChannel = ( ChannelSftp )channel ;

            success = true ;

        }

        catch( IOException e )

        {

            e.printStackTrace( ) ;

            success = false ;

            throw e ;

        }

        catch( JSchException e )

        {

            e.printStackTrace( ) ;

            success = false ;

            throw e ;

        }

        finally

        {

            System.err.println( "SECURELY connect status  " + host + " with user name " + userName + " :" + ( success ? "SUCCESS" : "FAIL" ) ) ;

        }



        return success ;

    }



    public void close( )

    {

        if( this.sftpChannel != null )

        {

            this.sftpChannel.exit( ) ;

        }



        if( this.session != null )

        {

            this.session.disconnect( ) ;

        }

    }



    public boolean downloadFile( String remoteFile, String localFile )

    {

        try

        {

            this.sftpChannel.get( remoteFile, localFile ) ;



            return true ;

        }

        catch( Exception e )

        {

            e.printStackTrace( ) ;



            return false ;

        }

    }



    public boolean uploadFile( String localFile, String remoteFile )

    {

        try

        {

            this.sftpChannel.put( localFile, remoteFile ) ;



            return true ;

        }

        catch( Exception e )

        {

            e.printStackTrace( ) ;



            return false ;

        }

    }



    public boolean deleteFile( String remoteFile, int tryCount ) throws IOException

    {

        int count = 0 ;



        while( count++ < tryCount )

        {

            try

            {

                this.sftpChannel.rm( remoteFile ) ;



                return true ;

            }

            catch( Exception e )

            {

                System.err.println( "Trial " + count ) ;

                e.printStackTrace( ) ;

            }

        }



        return false ;

    }