Jul 9, 2013

SQL Server T-SQL Tips & Tricks

Introducing

Today everyone is talking about NoSQ, but SQL is still an essential language that every programers should know enough about it as same as JavaScript and XML. MS SQL Server is a SQL engine that enables programmer to forward all data-related processes and worries to it, and just interact with it by SQL. now let's have a closer look at MS SQL Server.

Background

Because we are talking about tips here, so you need a good background of SQL ISO and MS SQL Server, not as a DBA, but have to know enough about DML, and DDL scripts.

Getting Started

We (Programmers) are usually have tight contact by their IDE (Eclipse, KDEV, NetBeans,...), in fact IDE is our best friend, but most of the times we need to persist our data and here the first guy we can think about it is SQL.
Each developer has a favorite (best-fit) database for saving the data (Derby, MySQL, PostgreSQL, Oracle, ... ), unfortunately most of the times we just run some commands to the server. We imagine a database is exact as same as ANSI SQL ISO definition (that should be, but who does follow the rules?).
Many of us just install and configure a login account in the DB and don't care about anything else. and what IF that database is SQL Server? so let's get started.
Table Of Content

xp_cmdshell

How to run CMD commands in SQL Server?

easy by using xp_cmdshell.
there is a good?! guy in MS SQL called xp_cmdshell, this procedure is used for running CMD commands, I think this is not really required while SQL server supports Managed Beans too, BUT I have to mention this procedure has defined before managed code introduced.

fortunately this procedure is disabled by default because of security concerns.

How to Enable xp_cmdshell in SQL Server?

this is going to be easy, at the first we need to enable advanced option with sp_configure procedure like this
--it doesn't really show anything advanced
sp_configure 'show advanced options','1';
reconfigure;--it's required
then after enabling advanced options and reconfigure the system, now it's time for enabling xp_cmdshell procedure
sp_configure 'xp_cmdshell','1';
reconfigure;
And now we have xp_cmdshell and he is ready for action, this procedure accepts two arguments, the first one is the process name with the input argument, and the second one is 'no_output'. 'no_output' means don't show the response of executed application.
exec xp_cmdshell 'echo %JAVA_HOME% , %INTELOCLSDKROOT%;'
exec xp_cmdshell 'echo %WinDir%;'
exec xp_cmdshell 'echo ''Hi Buddy'';','no_output'
The results of called commands are shown as result set in SQL (no messages).
the xp_cmdshell returns an integer number that it's result of run command
declare @res int;
exec @res=xp_cmdshell 'C:;','no_output'
if(@res=0) begin
  print ('Drive C is available :D');
   end
 else begin
  print ('Drive C is not available !!!');
 end
exec @res=xp_cmdshell 'F:;','no_output'
if(@res=0) begin
  print ('Drive F is available :D');
   end
 else begin
  print ('Drive F is not available');
 end

Limitations

1: you cannot run every all applications, generally terminal applications work without any problem but you cannot run a GUI application like notepad.exe
2: it waits(block) until the command you run response.
3: you cannot set or get application's stream directives.

Example

now let have a good example.

How to call BCP tool is SQL Server?

using xp_cmdshell.
The BCP, its a tool installed by SQL Server, it's used for copy data from SQL structure into user files. so because it's a executable application and it's not a SQL command we need to call it by xp_cmdshell
use master;
go;
create table v(id int identity(0,1),"name" varchar(64));
go
insert v values ('911'),('992'),('991'),('970'),('972'),('You');
declare @bcpComm varchar(512),@r int;
select @r=round(rand()*count(*),0) from v
set @bcpComm='BCP "select ''The lucky person is :....''+[name] from master.dbo.v where id=''"'+
     convert(varchar(10),@r)+''' queryout C:\users\lucky.txt -T -c -S'+@@servername
exec @r=xp_cmdshell @bcpComm,'no_output';
if(@r=0)print 'Success, please check the C:\users\lucky.txt file';
else print('Error!');
drop table v;

Exec() and sp_executesql

How to run dynamic query in SQL Server?

easy using sp_executesql procedure
Both exec() and sp_executesql are used for running dynamic SQL commands, just need to prepare the commands as a String object and call the sp_executesql.
Both of exec() and sp_executesql are same, exec() is the new face of sp_executesql has added in SQL Server V9. This procedures are enable by default.

How to dynamically create to many tables sequentially in SQL Server?

easy, using a while loop, prepare each DDL statement and run the command string by sp_executesql procedure.
The following example creates 100 tables dynamically.
declare @com nvarchar(1024);
declare @i int=0;
while(@i<100)begin
--step1: prepare the command
set @com='create table MyTable'+convert(varchar(3),@i)+'(id int,[name] varchar(64));';
--set @com='drop table MyTable'+convert(varchar(3),@i);
--step2: run the command
exec sp_executesql @com;
--exec(@com);--as same as above, the new face
set @i=@i+1;
end
--step3: verify the work
select * from sys.tables where name like 'MyTable%'

xp_regwrite and xp_regread

Is it possible to access registry in SQL Server?

Yes
While it doesn't make sense, but this is possible to read an write registry with SQL Server.
I'm wondering! SQL doesn't support arrays or partial foreign keys or hash index, or many things a good database needs, but it support access to registry, and I don't know when does a database programmer really need to access registry?! But HONESTLY I used this feature to much, because of many time my application couldn't access registry, SQL helped me out to much with this case.

Is it possible to SELECT the installation path in SQL Server?

yes, the installation path is located in registry.
declare @ipath nvarchar(256);
--SQL SERVER V100 path, change SQL10 to SQL9 for V90
exec master..xp_regread N'HKEY_LOCAL_MACHINE',N'Software\Microsoft\Microsoft SQL Server\SQL10\Setup' ,N'SQLPath',@ipath output,no_output 
print @ipath

Is it possible to start SQL Agent in SQL?

yes, find the installation path, and run the SQL agent process by command.
--step 1: find installation path
declare @ipath nvarchar(256);
exec master..xp_regread N'HKEY_LOCAL_MACHINE',N'Software\Microsoft\Microsoft SQL Server\SQL10\Setup' ,N'SQLPath',@ipath output,no_output 
--step 2: prepare the start command command
declare @sc nvarchar(1024);
--SQL agent is located under INSTALLATION_PATH/binn/sqlagent.exe
--set the server need to binds with SQL agent with -i parameter
set @cmdComm=N'start '+@ipath+'\binn\SQLAGENT.exe -i '+cast(@@servername as nvarchar(2048));
declare @rv int;
--execute the command line by xp_cmdShell
exec @rv=xp_cmdshell @cmdComm;
if (@rv=0)begin print 'Agent started successfully'; end
else begin print 'Agent initiating failed!'; end

Is it possible to enable TCP protocol in SQL Server by T-SQL?

Yes

how to enable SQL Authentication mode by T-SQL?

Set the (HKEY_LOCAL_MACHINE\Software\Microsoft\MSSQLServer\MSSQLServer)loginmode key to 2
--step 1: creating a login (mandatory)
create login login_to_system_after_injection with password='Thank$SQL4Registry@ccess';
GO
--step 2: enabling both windows/SQL Authentication mode
/*some server specific configurations are not stored in system (SQL)*/
--set the value to 1 for disabling the SQL Authentication Mode after . . .
exec xp_regwrite N'HKEY_LOCAL_MACHINE', N'Software\Microsoft\MSSQLServer\MSSQLServer', N'LoginMode', REG_DWORD, 2;
--step 3:getting the server instance name
declare @spath nvarchar(256);
--SQL SERVER V100 path, use SQL9 for V90
exec master..xp_regread N'HKEY_LOCAL_MACHINE',
                 N'Software\Microsoft\Microsoft SQL Server\Instance Names\SQL' ,N'SQL10',@spath output,no_output 
--step 4:preparing registry path
declare @insRegPath nvarchar(1024)=N'Software\Microsoft\Microsoft SQL Server\' + 
                                      @spath + '\MSSQLServer\SuperSocketNetLib\Tcp';
--step 5:enabling tcp protocol'
exec xp_regwrite N'HKEY_LOCAL_MACHINE', @insRegPath, N'Enabled', REG_DWORD, 1 --generally tries to enable all addresses. NOT Recommended
--step 6:enabling remote access
EXEC sys.sp_configure N'remote access', 1
GO
RECONFIGURE WITH OVERRIDE --reconfigure is required!
GO
--step 7:a system restart is required in order to enabling remote access.
--step 7.1:shutting down the server
shutdown
--After this command you need to start the server implicitly yourself.
--or just configure the Agent in order to start the server at any shutdown or failure 

Cursor

What is cursor in SQL Server?

Cursor is SQL data-type, this is used for saving a select result in order to seek through rows.
Cursor is a really useful guy, specially when you want to fetch records one-by-one, generally most of the database systems support cursor because it has defined in SQL ISO.

How to declare a cursor in SQL Server?

cursors are declared as same as variable declarations with a small difference, cursors variable doesn't follow variable name in SQL, there is no @ symbol at first of cursor variables.
--forward-only cursor declaration, can just get the next result(record)
declare cur cursor for select * from dbo.a
--scrollable cursor declaration, can move freely through cursor
declare cur scroll cursor for select * from dbo.a
Unlike explicit transactions, by default cursors are alive until you deallocate(remove) them by deallocate command
--removing cursor c completely, it's better close it before deallocation
deallocate c;
for using a cursor, you need to open the cursor, fetch information and close it. A cursor has a pointer that points to the current row in the result, just after you open a cursor, pointer points to nothing (before first), so for fetching the first row, we need to fetch the next result(move the pointer). the global @@fetch_status variable is used for determining the current status of the cursor
  +------------------------------------------+
  |--> Before the result @@fetch_status = -1 |
  |-> D A T A - record  @@fetch_status = 0   |
  |-> D A T A - record  @@fetch_status = 0   |---\
  |-> D A T A - record  @@fetch_status = 0   |    \ A  Cursor
  |-> D A T A - record  @@fetch_status = 0   |    /  Context
  |-> D A T A - record  @@fetch_status = 0   |---/
  |-> D A T A - record  @@fetch_status = 0   |
  |--> After the result  @@fetch_status = -1 |
  +------------------------------------------+

Example

The following example tries to show folks table records in a specific format, as above diagram is showing, when a cursor get opened, the pointer is just before the first result, so we need to fetch next from the cursor, then iterate the cursor by a loop.
create table folks(id int identity(1,1),name varchar(32));
go
insert folks values('911'),('918'),('5780'),('718');
--step1:declare the cursor
declare cur scroll cursor for select * from folks;
declare @id int,@name varchar(32);
--step2:open the cursor
open cur;
--step3:fecth the very first record into variables 
fetch next from cur into @id,@name
print('--------------------------')
--step4:read the cursor to end
while( @@FETCH_STATUS=0 )
begin
 print(convert(varchar(10),@id)+space(10)+@name);
 fetch next from cur into @id,@name;
end
print('--------------------------');
--step5:close the cursor (close the handler)
close cur;
--step6:finilize the cursor
deallocate cur;
drop table folks;

Always use cursor in triggers

What is a trigger?

Trigger is a functional component which is fired implicitly by a DML or DDL event, it's used for doing alternatives with an specific event.
When you have a trigger on a DML operation, this is possible that user inserts or deletes more than 1 record by one insert call, so in the trigger you should have a pattern to check all of the records affected, that this could be done very easy by trigger.
create table folks(id int identity(1,1),name varchar(32));
go
insert folks values('Apache'),('CodeProject'),('Google'),('sourceforge'),
       ('Eclipse'),('IBM'),('Oracle'),('Valve');
go
--step1:creating the trigger
create trigger no_close_src on folks for insert as begin
--step2:creating the cursor
--local keyword here ensures you the cursor is accessible just this block(procedure)
declare insData cursor local for select name from inserted;
declare @name varchar(32);
open insData;
fetch next from insData into @name;
--step3:while it contains records
while(@@fetch_status=0) begin
 set @name=lower(@name);
 --step4:check
 if ( @name='microsoft' or @name='adobe' or @name='autodesk' or @name='ea') begin
  print('Error, there is no room for close source guys('+@name+') here!');
  rollback transaction;
 end
 --step5:fecth the next value and check it too
 fetch next from insData into @name;
 end
end
go
insert folks values('github'),('ea'),('Sun');

select * from folks;

drop table folks;

instead of triggers

is it possible to override a DML/DDL operation in SQL Server

yes, using instead of triggers.
Just create an instead of trigger on target object for desired event(s), but SQL doesn't (instead of) support it for every DDL operations of course.

How to avoiding insertion in a table without rollbacking the transaction?

Using instead of.
create table empty_4_ever (id int identity(0,1),_name varchar(128));
go
create trigger overrideInsert4Empty4Ever on empty_4_ever instead of trigger as begin
    --there is nothing to do
    --because this is an instead of trigger so there is no any transaction, no any insertion by db
    print('are you really sure you have inserted data?');
    --if this have insert in empty_4_ever table, this is mean an actual insert into the table without calling itself again
end
go
insert empty_4_ever values('we are wasting our time'),('have a try if you cannot believe it');
select * from empty_4_ever
As mentioned instead triggers in SQL are like method overriding in Java, there is actual insert operation behind, but when you call the method, you are calling the overridden method, there is just one chance to invoke the original method/command, and this is overridden method calls the base method itself.


Click to enlarge

this story is same for instead of triggers in SQL server too. the following list is showing the key notes about the instead of triggers.
  • Because there is no any actual command performed by db, so there is no any transaction to commit or rollback.
  • If an instead of trigger calls its event again, it doesn't call itself again, the actual event will be invoked, for example an instead of trigger on table guys on insert, if inserts a value in table guys, it doesn't call itself again, it calles the actual insert.
  • There is only ONE instead of trigger is allowed for each event on a member
  • if there are 3 trigger(for, after, instead of ) on a member on same event(s), then the first firing trigger will be instead of trigger, other triggers will no fired UNLESS instead of trigger repeats the event.
  • Use instead of if the whole of event process should overridden, or the rate of rollbacking data is to much high.

Multi-Threading in SQL Server

Is is possible to run threads in SQL Server?

yes, but not as same as you do in Java or C, using Service broker

What does service broker do in SQL Server?

service broker is a semi-implementation of Message Oriented Architecture(MOA)

What is MOA?

it's a pattern(architecture) designed for heterogeneous to communicate with each other. The main MOA features are listed below:
  • Sending sync or async messages to the target server
  • Holding events/messages in safe to prevent removing by system crashes
  • Scheduling to invoking a function in future
  • Sending messages across heterogeneous systems
And there is just one big disadvantage, and this is [it's unreliable] because there is no any certain method to monitor current state of a request in server from the client. SMS is an example of MOA.

What are a MOA's components?

Generally a MOA system contains component listed below, but it's not a rule, MOA is architecture, is a definition. so there is no any rule defined to force every MOA implementations to have these components. for instance SQL Server doesn't have Topic and agreement.
  • Server: the main application, MOA implementation
  • Topic & Queue: are used for holding messages
  • End Point(service): the target function needs to be invoked
  • Agreement: is the protocol used for communication
  • Message type: the data-type for storing data, usually is typed XML
  • Contract: method for marshaling and unmarshaling message in order to transfer in network
in SQL server we just need to create message type, queue, service, contract,and. as I said create, no implementation, it's really simple.


Click to enlarge

How to execute a procedure parallel in SQL server?

Simple, as following order.
  1. Activate the service brokers in both client and server points(in this example both client and server are same)
  2. Create a procedure for handling request in server
  3. Create Message type and contract in both client and server points
  4. Create the server queue and bind the procedure to
  5. create the server service and bind the service to it
  6. create the queue in client
  7. create the service in client for sending messages
  8. create a dialog conversation(session) and send message through session
/*At the server*/
create database _parallel
go
use _parallel
go
--step1: create the procedure (thread)
create proc threadStart as begin
declare @comm nvarchar(2048);
--getting message from the messageCommPool queue
receive top(1) @comm=convert(nvarchar(2048),message_body) from messageCommPool;
--execute the message as sql command, or anything else
exec(@comm);
end
go
--step2: enable service broker
alter database _parallel set enable_broker
go
--step3: define the message data-type
create message TYPE messageComm
validation = none
go

--step4: Create the receiving queue
create queue [messageCommPool] with status=on, retention=on ,activation (
        procedure_name = _parallel.dbo.threadStart,
        max_queue_readers = 32 , --maximum instance of parallel invocation
        execute as owner)
go
--step5: create the contract
create contract theContract
(messageComm SENT BY any)
go
--step6: Create Receive Service on Recieve Queue
create service hostService
on queue messageCommPool (theContract)
go


/*At the client*/
/*
Create message type and contract IF and IF client is located in another server or db
use clientDB;
go
alter database clientDB set enable_broker
go
--step3: define the message data-type
create message TYPE messageComm
validation = none
go
--step5: create the contract
create contract theContract
(messageComm SENT BY INITIATOR)
go
*/
--step7: create the sending queue at client
create queue sendQueue with status=on, retention=off;
go
--
create service sendService
on queue sendQueue (theContract)
go
--step7: creating a procedure to sending message to teh server
create proc runThread (@comm nvarchar(2048)) as begin
 --step8: initilizing the connection session
 declare @session uniqueidentifier
 --step9: open the connection
 begin dialog conversation @session
 from service sendService
 to service 'hostService'--hostService is the service name located at the target server
 on contract theContract
 with encryption = off;
 --step9: send the command
 send on conversation @session
 message type messageComm (@comm);
end
go
--step10: have a try! :D
begin
declare @i int=0,@comm nvarchar(2048);
while(@i < 10)begin
 set @comm=N'create table tableNumber'+(convert(nvarchar(2),@i))+'(id int identity(0,1),[name] char(16));'
 exec runThread @comm;
 set @i=@i+1;
 end
end
--step11: verify the try
select * from sys.tables

/*
--step12: enything else?
use master
go
DROP database _parallel
go
*/
Service broker is a really nice feature in SQL Server, very useful, but SQL Agent would does the same as scheduling jobs, and you have better control on functions.

Managed Code

Is it possible to run a non-sql command in SQL?

Yes, using managed codes.
SQL Servers supports utilizing .Net platform in order to SQL-Unsupported and complex codes, for example compressing a file, or connecting to a URL, or killing a process.
We just need to implement the target with a CLR language(C#, VC++, VB) and just create a procedure in SQL Server that points to the actual code, so the .Net modules will be invoked by SQL procedure call.
This future is disabled by default, so first .Net communication should activated in the server.
exec sp_configure 'clr enabled', 1
go
reconfigure;
go
Then code the business in .Net
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.IO;
namespace wasys911992.blogger.example.sql{
public class StoredProcedures{
 //defining the method as a SQL proc
    [SqlProcedure()]
 //the method signaturie should be as same as procedure signature
    public static void writeToFile(SqlString filePath, SqlString data){
  //write content to the file
        BinaryWriter bw=new BinaryWriter(new FileStream(filePath.Value,FileMode.Create));
  bw.Write(data.Value);
  bw.Flush();
  bw.Close();
  //accessing sql pipe for sending message, works as print in sql
  SqlContext.Pipe.Send("Success! hello from PRAHS-C");
    }
  }
}
Then compile the class, and get the .dll or .exe file.

So there is a problem, by default, SQL Server and .Net doesn't trust the code, because it may harm the system, so by default a managed code cannot access external resources, and would just get some value and return some value, but in the above code, we tried to save the content in a FILE, so it means the code is trying to access a external resource here. this kind of codes are called unsafe while their unsafe too.

How to run unsafe managed codes in SQL Server?

Easy, just tell the SQL be relax and trust every one, no enemies here.
-- tell sql take it easy, why so restriction?! be open
alter database mydb set trustworthy on
Okay, time to register your assembly (.dll) file in SQl server with unsafe permission.
create assembly myAssembly authorization dbo
from 'C:\managed\managed.dll' with permission_set = unsafe
Okay and now, it's time to create a procedure and point to the actual method in the assembly.
create procedure writeToFileProc (@name nvarchar(1024),@data nvarchar(4000)) as external name
myAssembly.[wasys911992.blogger.example.sql.StoredProcedures].writeToFile;
and finally have a try, let's have all together
--step1: trust to everyone
alter database mydb set trustworthy on
go
reconfigure
go
--step2: register the assembly in sql
create assembly myAssembly 
from 'C:\managed\managed.dll' with permission_set = unsafe
go
--step3: create the procedure
create procedure writeToFile (@filePath nvarchar(1024),@data nvarchar(4000)) as external name
myAssembly.[wasys911992.blogger.example.sql.StoredProcedures].writeToFile;
go
--step4: have a try
exec writeToFile N'C:/data.txt',N'Hello from managed codez, :D';
Except procedure, trigger, function, and data-type would be signed as managed code too, but DON'T use it if you DON'T need it, SQL is heavy enough, don't make it heavier with .Net.

Tips

Use one insert command to insert multiple records

Never use 10 insert command for 10 record insert, fortunately after years and years and years Microsoft finally provided multiple row insert with one insert command implicitly, while just before 2008 select-union all approach had used.
--never use this approach :(
insert guys values( 1994,'911')
insert guys values( 1996,'718')
insert guys values( 1987,'970')
insert guys values( 1995,'918')
insert guys values( 1997,'5780')
insert guys values( 1993,'914')
insert guys values( 1997,'944')
--use this instead in V100 and higher
insert guys values( 1994,'911'),( 1996,'718'),( 1987,'970'),
                      ( 1995,'918'),( 1997,'5780'), ( 1993,'914'),( 1997,'944')
--use this for V90 and lower
insert guys select 1994,'911' union all
            select 1996,'718' union all
            select 1987,'970' union all
            select 1995,'918' union all
            select 1996,'5780' union all
            select 1993,'914' union all
            select 1997,'944' union all
10 inserts requires 10 transaction, while 1 insert needs one, in other word, one big load is better than to many small ones.

How to prevent inserting empty string in SQL Server?

Setting a field not null doesn't mean this field doesn't accept blank strings too! so for preventing empty strings, beside not null definition just add a check constrain to check the empty ones.
create table t(n varchar(64) not null ,constraint chkn check (ltrim(n)!=''));
insert t values('914 (this is okay!)'),
  ('   918 (this is okay too!)'),
  ('        Code Project folks(this is still okay!)!      '),
  ('    '/*Not okay!*/),(null /*Not okay!*/ );
simple remove the spaces with ltrim() function, and check the value should not be as a empty string ''.

How to insert single-quote character in SQL Server?

Simple, two single-quote together means one single-quote in SQL server.
insert messages values('Hi I''m a big fan of ''Java!'' Yeah''');

Encrypt critical data

SQL Server supports encryption/decryption algorithm in order to secure important data, this is very useful when a database should get shared among developers, so this ensures you no one will able access the encrypt data UNLESS he got the password!
create database secureGuys;
go
use secureGuys;
go;
--for encrypting a data, a binary field is required.
create table DavidsFans(id bigint identity(-1,1),data varbinary(2048) not null);
go
--step1:create the key
create symmetric key theKey with algorithm=AES_256  encryption by password='P!nk-FloyD-PULSE-1994';
go
create trigger new on dbo.DavidsFans instead of insert as begin
 --saving inserted row(s) in a cursor
 declare C cursor local for select data from inserted;
 open C;
 declare @data varbinary(2048);
 fetch next from C into @data;
 --step2: open the symetric key for encrypting data
 open symmetric key theKey decryption by password='P!nk-FloyD-PULSE-1994';
 --while we have row
 while(@@fetch_status=0)begin
 --step3: encrypt the data with encryptbykey function
 set @data=encryptbykey(key_guid('theKey'),@data);
 --step4: insert the encrypted data instead of pure data
 insert DavidsFans values(@data);
 fetch next from C into @data;
 end
 --step5: close the handler of key
 close symmetric key theKey;
 --close and destroy the cursor, it doesn't required if "close cursor on commit" has enabled
 close C; deallocate C;
end
--step6: have try
insert DavidsFans values(cast('Me, 911992' as varbinary(2048))),
      (cast('Me Again' as varbinary(2048))),
      (cast('you' as varbinary(2048)));
--step7: have look at encrypted data, how do they look?
select * from DavidsFans
go
--a proceure for showing the data
create proc _fetch  as begin 
--step8: open the key again
open symmetric key theKey decryption by password='P!nk-FloyD-PULSE-1994';
--step9: now decypt the data by decryptbtkey function
select *,convert(varchar(2048),DecryptBykey(data)) as 'Real Data' from DavidsFans
--step10: close the key handler
close symmetric key theKey;
end
go
exec _fetch
go

Indexes, friend or foe?

We all know index improves select fetching performance, but beside it's really helpful for DQL(select) but it's just like a daemon(foe) for DMLs, because when you have insert, you have one insert in physical table, and one insert into each index. and the worst part is, inserting in indexes needs to find the exact value then insert, so this is going to be very huge process.
I AM NOT saying don't use indexes, I want to say remove or disable indexes when you have a bulk update or bulk copy, just check the following example and see yourself.
the following code is about a simpel table that doesn't have any non-clustered index, we are trying to insert 10K records with randomized value into it, how does it may take?
create table folks(id bigint identity(0,1) primary key,
       "name" char(8), age smallint,rate float);
go
--truncate table folks
--getting start time
declare @stratTime datetime=getdate();
declare @i int=0;
declare @rand float;
--we are trying to insert 10K records, where there is no any non-clustered indexes
while(@i<10000) begin
 set @rand=rand();
 insert folks values(convert(char(8),round((@rand*9998),0)),
      convert(tinyint,@rand*255),
      @rand);
 set @i=@i+1;
end
print('without indexes, 10K records got '+convert(varchar(64), datediff(ms,@stratTime,getDate()))+' ms :D, good enough');
as you see, without indexes , DMLs are fast as Rocket, but what if we have indexes here?
truncate table folks
create nonclustered index nameIdx on folks("name" desc);
create nonclustered index ageIdx on folks(age asc);
create nonclustered index rateIdx on folks(rate);
go
--getting start time
declare @stratTime datetime=getdate();
declare @i int=0;
declare @rand float;
--we are trying to insert 10K records, where there is no any non-clustered indexes
while(@i<10000) begin
 set @rand=rand();
 insert folks values(convert(char(8),round((@rand*9998),0)),
      convert(tinyint,@rand*255),
      @rand);
 set @i=@i+1;
end
print('with indexes, 10K records got '+convert(varchar(64), datediff(ms,@stratTime,getDate()))+' ms :(');
This is not belong to SQL Server individually, this is belong to DBMSs, so just beware when you have some big DML task, it's better to either disable or drop indexes, then recreate/enable them after the bulk task.

SQL Injection

SQL Injection, try to run a command by values, in injection, you don't have any access to the target database, or you don't know even what is going on there!? what is database vendor? how many tables? how many objects? nothing, for example, injection would be appeared in a simple login form, for example, check this simple login form below.

Okay, now we should looking for possibilities, for example we have to think about the possible command that checks entered username and password, for example it could be like this.
select userId from users where username= '{Username Value}' and password= '{password Value}';
While it never going be like above, but any way, now you have a chance to inject your code with very last value, as above code, the password value is last thing in the query, so what if we enter " anything' or 1=1 ;-- " ? so the target generated command will be like this
select userId from users where username= 'Fender' and password= 'anything' or 1=1 ;--';
So we could inject our code easily, because while password is 99.9% in false, but 1 is equal to 1 every time, so you logged in to the system without any problem, BUT DON'T BE SURPRISED, this issue has been solved in many applications today, BUT you still have chance to search through beginners have just established their website, and have some fun, but try to learn, instead of destroy.

Nothing more well SQL Server is a good database, but if you ask me, I prefer PostgreSQL, this is true, SQL Server is enterprise and has a big support and development team, You and I as a developer except the features related to the database from a database, while SQL supports web services but who really is going to implement its web service with SQL Server? it's really bad while SQL Server doesn't support arrays, modern join commands and algebra, maps, hash indexes, inheritance, and, and and ..., BUT it supports registry access and web services.
so have a good SQL Server database.

Jul 3, 2013

Java Thread Example

Introducing

This article is about a few simple java thread examples for beginners, each example has an use case(business) and the solution. It's better to have review about java threading basis before you attempt to read this article.

Getting Start

This post contains some case study about multi-threaded applications. These examples are really easy, they may need too much code, but the definition is easy and there is no room for being worry.
Table of Content

#0 Type It

Simply, user has got 5 attempt chances to type down a sentence by std::in, simultaneously a thread is run to count the elapsing time.

package wasys911992.blogger.example.thread;
import java.io.BufferedReader;
import java.io.InputStreamReader;
/**
 * by github.com/911992
 * arashmd.blogspot.com
 */
public class Timer implements Runnable {
  public void run() {
    int c = 0B0;// int var= 0B<<binary value here>>, 0X<<hex value here>>, 0<<octed value>>
    try {
      while (!Thread.currentThread().isInterrupted()) {
        c++;
        Thread.sleep(1000);
      }
    } catch (InterruptedException ex) {
    } finally {
      System.out.println("Time elapsed: " + c + " seconds");
    }
  }
  public static void main(String[] args) throws Exception {
    BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
    Thread timer_thread = new Thread(new Timer());
    int att = 1;
    System.out.println("Type \"Victory is ours\"");
    timer_thread.start();
    while (!(br.readLine()).trim().equalsIgnoreCase("Victory is ours") && att < 5) {
      System.out.println("NOPE! wrong input, type \"Victory is ours\"");
      att++;
    }
    timer_thread.interrupt();
    if (att < 5) {
      System.out.println("Perfect you could type a simple sentence after "+(att)+" attempts");
    } else {
      System.out.println("Ehh... you attemped 5 times to type down a simple sentence and couldn't!");
    }
  }
}

Explanation

The main thread starts the timer thread and gets inputs from user. Either if user inputs the correct or wrong answer within 5 attempts(chance), main method will interrupt the timer thread indicates the stop signal for.

#1 Big File Maker

This example is about an application tries to make big files in a folder, but concurrently, it makes 10 large file at a same time, this kind of application is not really useful anymore, it more looks like a disturbing application, but it would be fun, for example, filling up whole of your memory flash disk (Don't do that).
We have one implementation of the task because all of the them follow a rule. As system needs to run a task many times, then we run 10 instance of implementation concurrently by threads.
package wasys911992.blogger.example.thread;

import java.io.DataOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
public class Base {
  /**
   * by github.com/911992
   * arashmd.blogspot.com
   */
  public static void main(String[] args){
    for(int i=0;i<10;i++){
      new Thread(new BadThread("./output/"+i+".text")).start();//any path you like, would be /home/user or C:/
    }    
  }
}
class BadThread implements Runnable{
  public BadThread(String filePath) throws IOException{
    File f=new File(filePath);
    if(f.exists()==false){f.createNewFile();}
    dos=new DataOutputStream(new FileOutputStream(filePath));
  }
  private DataOutputStream dos;
  double val=0.0D;
  @Override
  public void run() {
    try{
        for(int i=0;i<1024000;i++){//it could be even more 
          val=Math.random();
          for(int j=0;j<8;j++){
            dos.writeDouble(val+j);//there is no buffering [0]
          }
        }
        dos.flush();
        dos.close();
    }catch(Exception e){return;}
  }
  
}

Explanations

The main class(thread) creates n number(10 here) instances of BadThread, which each instance accepts a string indicates the target file path to be created.
You may change the number of threads, the path, or the size need to be created.

#2 Array Initializing

Considering a simple game having too many objects. It results too much processes for calculating object's location and rendering.
A simple or complex math formula would be used for calculating and rendering an object in the game, free fall for example, or throwing an object. We use two formulas here for instance, one is f(x)={x*x} and another one is f(x)={|X|*2}.
Consider the above diagram, it's talking about a 10-length array that each index value should multiply by itself(power) and gets stored into naother array, so here for 10 values, we run 10 threads. Each thread calculates a value.
package wasys911992.blogger.example.thread;
import java.io.DataOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;

//this is our formula signature
interface Function{
  Double calculate(double arg);
}

class PowerFunction implements Function{//a sample of formula
  @Override
  public Double calculate(double arg) {
    return Math.pow(arg, 2);
  }
}

class X2Finction implements Function{//another sample of formula
  @Override
  public Double calculate(double arg) {
    return Math.abs(arg)*2;
  }
}

public class GraphPointer{
  static int resCount=0;//indicates the number of finished job(thread)
  //number of values need to process, here we are talking about sequential numbers, but could be in any order
  static final int totalValue=360;
  static DataOutputStream out;//for saving the response to file, stream
  static Double[] res;//array to keep result
  /**
   * by github.com/911992
   * arashmd.blogspot.com
   */
  public static void main(String[] args) throws Exception{
    GraphPointer.out=new DataOutputStream(new FileOutputStream("C:/output/dataRes.txt"));
    //create a function of a formula, power, 2x, etc...
    Function f=new PowerFunction();
    //init the result array
    res=new Double[GraphPointer.totalValue];
    //for each element in array
    //we sued the -180 -> 179 numbers, this not a rule, could be in any order
    for(int i=0,val=-(res.length/2);i< res.length ;i ++,val ++){
      //run thread for each 
      new Thread( new FunctionInvoker(val, f, i) ).start();
    }
  }
  
  
  public synchronized static void addResultSet(){
      GraphPointer.resCount++;
      //if all of the results have calculated
      if(GraphPointer.resCount==GraphPointer.totalValue){
        //persist the data
        GraphPointer.persistData();
      }
  }
  
  //this method gets called when all of results get calculated
  private static void persistData(){
    try{
      //here it writes doubles as string, could be double or in any format
    for(Double d : res){out.writeChars("["+d.toString()+"]  ");}
    out.flush();out.close();}catch(Exception ex ){ex.printStackTrace();}
    System.out.println("DONE!");
  }
  
  public static void setData(int i,Double val){
   res[i]=val;//set the calculated value into the desire index
  }
}

//this guys runs the functions as thread
class FunctionInvoker implements Runnable{
  Function f;//reference of function(formula)
  double arg;int i;
  public FunctionInvoker(double arg,Function f,int i){this.arg=arg;this.f=f;this.i=i;}
  @Override
  public void run() {
    //get the result from the function and pass it to get persist in array
    GraphPointer.setData(i,f.calculate(arg));
    //tell system that a result has been done.
    GraphPointer.addResultSet();
  }
}

Explanations

As you see in above code, the main thread just runs 360 threads concurrently. Results(y points) are stored in an array(res) that each thread saves its result to the related index.
There is an issue, after running threads the array data should persist in a file, but the problem is here, how to realize all threads returned the result? we didn't use of either semaphore or pool here, but we count the results, it's not really good idea, but it's a way to identifying how many threads has finished.

static setData(i:int,val:Double):void

this method is called when a thread calculates a result then the thread passes the index number and the result to it for storing the data. you may ask now why isn't it synchronized? why doesn't it lock the array to saving data? because we sure this is impossible that two threads calculate one index, definitely each index of the array is belong to one thread, also there is no any reading unless all of the array indexes initialized.

synchronized static addResultSet():void

This is called by setDate() method to count the results in order to persist the array once all the results are ready(calculated). It locks the resCount(indicates the calculated results) for updating because this is a read-modify update.

#3 Array Lookup

Considering a big data array or context such as a big database index page, and finding some data is required. For example counting the prime numbers ranged 0 to 16777216, or finding strings which contain spongebob.
We would see the whole array as multiple small arrays, for example considering a 16777216-len array as 32 x 524288-len arrays. Then simply each array will be looked up by a thread, something like the following diagram.

package wasys911992.blogger.example.thread;

//import java.util.ArrayList;

/**
 * by github.com/911992
 * arashmd.blogspot.com
 */
public class IndexSearch {
  static final int data[]=new int[16777216];//our data
  static{//Initializing the data linear(single-thread)
    long st=System.currentTimeMillis(); 
    for(int i=0;i<data.length;i++){
      data[i]=i;
    }
    st=System.currentTimeMillis()-st;
    System.out.printf("Initilization done | elapsed time: %d ms\n",st);
   }
  static final Object lock_obj=new Object();
  //there are 1077871 prime numbers between 0 to 16777216
  //static ArrayList<Integer> indexes=new ArrayList<Integer>(1077871);//for persisting the prime numbers 
  static volatile int total_primes=0;
  public static void main(String[] args) throws Exception {
   long st=System.currentTimeMillis();//get start processing time
   for(int i=0;i<data.length;i++){//run the process linear
     if(is_prime(data[i])){total_primes++;/*indexes.add(i);*/}//check the data
   }
   st=System.currentTimeMillis()-st;//get and calculate the elapsed time
   System.out.printf("Single thread | elapsed time: %d ms\n",st);
//   indexes.clear();
   total_primes=0;
   
   Thread[] guys=new Thread[32];
   int each_len=data.length/guys.length;
   st=System.currentTimeMillis();
   for(int i=0;i<guys.length;i++){
     guys[i]=new Thread(new SearchWorker(i*each_len, each_len));
     guys[i].start();//run each part 
   }
   for(int i=0;i<guys.length;i++){
     guys[i].join();
   }
   st=System.currentTimeMillis()-st;
   System.out.printf("Parallel(multithread) | elapsed time: %d ms\n",st);
  }
 static boolean is_prime(int a) {
   if (a%2==0) return false;
   for(int i=3;i*i<=a;i+=2) {
       if(a%i==0){ return false;}
   }
   return true;
 }
}
class SearchWorker implements Runnable{
  int offset,len;
  public SearchWorker(int offset, int len) {
//    System.out.println(offset + " "+ (offset+len));
    this.offset = offset;
    this.len = len;
  }
  public void run() {
    for(int i=offset;i<(offset+len);i++){
      if(IndexSearch.is_prime(IndexSearch.data[i])){
        synchronized (IndexSearch.lock_obj) {
          IndexSearch.total_primes+=1;
          //IndexSearch.indexes.add(i);
        }
      }
    }
  }
}

Explanation

Try the code yourself and see the differences. I got 5 seconds for linear(single-thread) process, and 800ms as parallel, so such a big difference. And what if I told you this time would be decreased to 10 ms by running the code via GPU? Yes, small and too much processes are just GPUs' breakfast.
Here we divided(assumed) the data array into 32 smaller array, and run a thread for it.

#4 File Encryption

Well there are too many methods for encrypting a file, but we could have our own method, here we want to encrypt files with this customized algorithm.
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfj8ckAURaeySuh8eX9s3hHs5QIgsjIrfmLjqS56TZW4AISNL4nJbQcY9XDtvjFXpb5AKX0LNQPouxM995dw63h5M7Y2M48njOPe_JdKa0Pv7czXOlERwrCf_KxYpX_O1P4XaIaNc67e4/s320/encryptdiagram.jpg
It's just a simple method that swaps data in a 16 length array, it could be in any order(as you wish), but if you are going to encrypt your data by your own method then note the algorithm you will use, or your data will be inaccessible.
Okay as you see in above diagram each instance swaps(encrypt) a 16-length byte array, so if raw file contains 16000 bytes of information, then we need 1000 instance(threads/calls) of encrypt module. We would run multiple calls(modules) parrallel, but 1000 threads at a time doesn't sound fair for CPU(not same for GPU), the we would limit the maximum of threads at a time with either a semaphore or a thread pool (you can find the tutorial here).
We use a customized thread pool.
package wasys911992.blogger.example.thread;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.ArrayList;
import java.util.List;
public class Encrypt {
  /**
   * @param args
   * @throws IOException 
   * @throws InterruptedException 
   */
  public static void main(String[] args) throws IOException, InterruptedException {
    FileInputStream fis=new FileInputStream("c:/input.txt");//input file either encrypted or raw file
    Encrypt e=new Encrypt();
    //set the encryptMode to false for decryption, true for encryption
    //e.encryptMode=false;//note that set it false when you are decrypting an encrypted file(no normal file) [0]
    byte[] data=new byte[16];int res;//here it reads 128 bits in each block
    int sequence=1;
    long st=System.currentTimeMillis();
    while(true){
      try {
        res=fis.read(data); //read 16 bytes from the raw/encrypted file(stream)
        if(res==-1){break;}//if End of stream (EOS) reached
        if(res==16){//if this is a complete block
          e.run(data, sequence);//pass the data and the sequence number to the thread pool
        }else{//if it's a incomplete block, so array copy is not needed for the last one.
          e.run(data, sequence);
        }
        sequence++;//next sequence means, next block
      } catch (Exception ex) {
        ex.printStackTrace();return;
      }
    }
    e.join();//waits until every running threads finishes its job(encrypt) [1]
    e.shutdown();//shutdown the threads(pools)
    System.out.print("File encryption/decryption finished! time: "+(System.currentTimeMillis()-st)+" ms");
  }
  public boolean encryptMode=true;//by default, system encrypt the file
  private List<EncryptModule> readyThreads, busyThreads;//list of ready and running threads [2]
  int maxThreadSize=64;//maximum size of running thread at a time, would be more or less 
  Writer w;
  private Encrypt() throws IOException{
    try {
      w=new Writer("C:/output/res.txt");//set the output file
    } catch (FileNotFoundException e) {}
    readyThreads=new ArrayList<EncryptModule>(maxThreadSize);
    busyThreads=new ArrayList<EncryptModule>(maxThreadSize);
    for(int i=0;i<maxThreadSize;i++){
      EncryptModule em=new EncryptModule(w,this);
      new Thread(em).start();//starts each thread [3]
      readyThreads.add(em);//filling the readyThreads(pool) [4]
    }
  }
  private synchronized void run(byte[] data,int sequence) throws InterruptedException{ //[5]
    if(readyThreads.size()==0){this.wait();}//if there is no thread ready, so wait for one! [6]
    EncryptModule em=readyThreads.get(0);//get a ready thread
    readyThreads.remove(0);//remove it from the ready ones
    busyThreads.add(em);//add it to busy pool
    em.newTask(data, sequence);//pass the block to process [7]
  }
  public synchronized void backToMgr(EncryptModule em){//called by EncryptModule when its job get finished [12]
    busyThreads.remove(em);//remove the thread from busy ones
    readyThreads.add(em);// and set it as ready
    if(readyThreads.size()==maxThreadSize){//if all of the threads has back, time to shutting down the system
      synchronized (readyThreads) {
        readyThreads.notify();//notify the join [8] thread
      }
      }
    try{
      this.notify();//notify that a new ready thread has added to the pool [5]
    }catch(Exception e){}
  }
  void join(){
    synchronized (readyThreads) {
      if(readyThreads.size()<maxThreadSize){//if threads are still in action
        try{
          readyThreads.wait();//Waits until backToMgr notify that all threads have back //[16]
        }catch(Exception e){e.printStackTrace();}
      }
    }
  }
  void shutdown() throws IOException{
    w.close();//close the file handler
    for(EncryptModule e : readyThreads){
      e.shutdown();//kills every thread in the pool [9]
    }
  }
}

/*
 *1 2 3 4           F 9 C 8
 *5 6 7 8  ----\\   A 1 B E
 *9 A B C  ----//   5 4 G 2
 *D E F G           7 D 3 6
 *
 *Encrypt algorithm
 *        outData[0]= inData[14];   outData[1]= inData[8];     outData[2]= inData[11];    outData[3]= inData[7];
          outData[4]= inData[9];    outData[5]= inData[0];     outData[6]= inData[10];    outData[7]= inData[13];
          outData[8]= inData[4];    outData[9]= inData[3];     outData[10]=inData[15];    outData[11]=inData[1];
          outData[12]=inData[6];    outData[13]=inData[12];    outData[14]=inData[2];     outData[15]=inData[5];
====================================================================================================================
 *F 9 C 8            1 2 3 4
 *A 1 B E   ----\\   5 6 7 8
 *5 4 G 2   ----//   9 A B C
 *7 D 3 6            D E F G
 *Decrypt algorithm
 *        outData[0]= inData[5];    outData[1]= inData[11];    outData[2]= inData[14];    outData[3]= inData[9];
          outData[4]= inData[8];    outData[5]= inData[15];    outData[6]= inData[12];    outData[7]= inData[3];
          outData[8]= inData[1];    outData[9]= inData[4];     outData[10]=inData[6];     outData[11]=inData[2];
          outData[12]=inData[13];   outData[13]=inData[7];     outData[14]=inData[0];     outData[15]=inData[10];         
 */
class EncryptModule implements Runnable{//encryption module, each instance has a 128 bits data [10]
  private Writer w;int sequence;Encrypt mgr;
  public EncryptModule(Writer w,Encrypt e){this.w=w;this.mgr=e;}
  byte[] inData,outData;Thread thisThread;
  private volatile boolean shutdown=false;
  private Object lock=new Object();
  @Override
  public void run() {//starting thread at the creation time [3]
    thisThread=Thread.currentThread();//obtaining the current(this) thread
    synchronized (lock) {
     while(this.shutdown==false){
      try{
        lock.wait();//waits for a new task... [13]
        if(inData.length==16){//if it's about processing a complete(128 bits) block
          outData=new byte[16];
          if(mgr.encryptMode){//if we need to encrypt
          outData[0]= inData[14];   outData[1]= inData[8];     outData[2]= inData[11];    outData[3]= inData[7];
          outData[4]= inData[9];    outData[5]= inData[0];     outData[6]= inData[10];    outData[7]= inData[13];
          outData[8]= inData[4];    outData[9]= inData[3];     outData[10]=inData[15];    outData[11]=inData[1];
          outData[12]=inData[6];    outData[13]=inData[12];    outData[14]=inData[2];     outData[15]=inData[5];
          }else{//if we need to decrypt
          outData[0]= inData[5];    outData[1]= inData[11];    outData[2]= inData[14];    outData[3]= inData[9];
          outData[4]= inData[8];    outData[5]= inData[15];    outData[6]= inData[12];    outData[7]= inData[3];
          outData[8]= inData[1];    outData[9]= inData[4];     outData[10]=inData[6];     outData[11]=inData[2];
          outData[12]=inData[13];   outData[13]=inData[7];     outData[14]=inData[0];     outData[15]=inData[10]; 
          }
        }else{//if it's about a incomplete block
          outData=new byte[inData.length];
          for(byte i=0, p=(byte)(inData.length-1);p>=0;p--,i++){//reverse the array
            outData[i]=inData[p];
          }
        }
        try {
          w.write(sequence, outData);//write the encrypted/decrypted data [11]
        } catch (IOException e) {
          e.printStackTrace();
        }
        mgr.backToMgr(this);//back to the ready pool [12]
      }catch(InterruptedException e){break;}
      }
     }
    }
  public void shutdown(){
    shutdown=true;
    thisThread.interrupt();
  }
  public void newTask(byte[] data,int sequence){//new task received [7]
    this.sequence=sequence;//set the new sequence number
    inData=new byte[data.length];//set the new block data
    System.arraycopy(data, 0, inData, 0, data.length);//copy the array to the local array
    synchronized (lock) {
      lock.notify();//signal this thread, that a new task have received [13]
    }
  }
}

class Writer{
  private RandomAccessFile raf;
  public Writer(String filePath) throws IOException{
    File f=new File(filePath);
    if(f.exists()==false){f.createNewFile();}
    raf=new RandomAccessFile(filePath, "rw");
  }
  public synchronized void write(int arrayNumber,byte[] data) throws IOException{
    raf.seek((arrayNumber-1)*16);//writes the data to the correct location of the file, 2nd. block, for 2nd. block
    raf.write(data);
  }
  public synchronized void close() throws IOException{
    raf.close();
  }
}
This program is used for both encrypting/decrypting files with our algorithm
Note: if you are planning to change the algorithm, just keep both of your encryption and decryption algorithm too(at least have a backup of the data you are going to encrypt), or your data will be lost.
A customized thread pool is used for this app. The main purpose of using a thread pool is limiting maximum number of running threads(like semaphore), and recycle(re use) of a thread resource(the main responsibility).

Explanations

Okay, let's have a closer look.
The Encrypt class is our thread pool here, but remember that it's not a real pool, because this is just designed in a such way to run EncryptModule classes, while thread pool should be completely independence of any businesses.
As mentioned, this application is used for both encryption and decryption, so all we want to specify is [0](find in code comments) to set encrypteMode variable to false for decryption.

Encrypt (class)

First this class creates a file for persisting encrypted/decrypted data, then initialize the pools (ready pool, and running pool).
By pool initialization all thread instances will be located in ready pool(state) and also get started [3], it means for a 32 length pool, we will have 32 ready (run) threads are ready for action.

synchronized run(data:byte[],sequence:int):void

This method is called by client application (here Encrypt class itself) when a new task need to get run by a thread[5], this method gets a thread from ready thread pool then removes it from ready pool and adds it to busy ones next passes the data array and sequence to the thread [7]. But if there is no any thread available(every threads is in action) then this method blocks [6] until a thread gets back to the pool.

synchronized backToMgr(em:EncryptModule):void

This method is invoked by a thread once finishes its work in order to head back to the pool and gets ready for next possible task[12]by removing the thread(em) from the busy pool and add it to ready pool. Next it tries to notify waiting clients thread blocked by run() method (if any)[6].

After reading whole of input file and the crypt process, it's time for shutting down the system, first we have to ensure about there is no any running thread, so it calls the join() method [1]

join():void

This methods waits(block) until all threads head back to the thread ensuring the system that there is no any running thread and it's safe for shutting down the pool. First it checks the size of the ready pool, and if it is filled, returns, else it will wait[16] on readPool object until all running threads head back, then returns. The backToMgr() method counts the readPool size for each call, and if it founds all threads are just in readyPool, then notifies the join[8].

shutdown():void

This method halts the thread pool by sending the shutdown signal to each thread in the thread pool [9]. The actual thread (EncryptModule) gets the shutdown signal and sets the shutdown flag to true and interrupts its thread in order to wake from the waiting state(if it is).

EncryptModule class

This class is our actual thread, its constructor gets a Writer instance(for persisting data) and an Encrypt instance heading back to the pool.
Two byte arrays are used, one for input data, and another for crypted data, and the lock Object is used for signaling for new tasks.[13]
This thread gets started by thread pool (Encrypt class)initilization. First it (the thread) waits for lock Object (notify signal)[13], then operates either encrypt/decrypt task, next passes the crypted data to Writer instance for persisting, finally heads back to the thread pool [12].

Writer class

This class uses a RandomAccessMemory(but it's better to use a direct file channel) for persisting data in order to able seek through the file and change the cursor because this is possible the 9th thread finishes its job earlier than the 8th or 7th ones.
This write operation is also synchronized(atomic) for preventing concurrent file write.
For more information about a simple thread pool, this simple Java Threading Tutorial would give a hand.

#5 Randomized Colorful Squares

This is very simple example, here we try to draw randomized color squares in an image using threads. Something like this.

or like this
This application generates 128x128-sized squares that contains 8 16x16 squares inside(each big square has a randomized color, and each small one has brighter or darker color), the squares size would be 64x64 or 32x32, as you wish, and the resolution is 768 x 512 (resolution should be dividable by squares size, both width and height).
For example if the resolution is 12 x 12, and the number of randomized color (big squares) is 9 (3 in column and 3 in row), and in each big square there are 4 small squares (2 width, 2 height), so we will have (12/3/2) = 2 pixel sized for each small squares.
We need an algorithm that identify and set each 2x2 pixel as a small square, but here we use a one-dimensional integer array for saving the whole big square.
Check the following example, it has got 9 big squire that each big one has 4 inside, so simply the first small squire indexes are 0,1,12,13.
01234567891011
121314151617181920212223
242526272829303132333435
363738394041424344454647
484950515253545556575859
606162636465666768697071
727374757677787980818283
848586878889909192939495
96979899100101102103104105106107
108109110111112113114115116117118119
120121122123124125126127128129130131
132133134135136137138139140141142143
Okay the code is not to much complex, in this one I tried to use a simple semaphore for limiting the maximum number of concurrent threads, but it will be nice by thread pool instead.
The config is: resolution = 768 x 512 || big squires size = 128x128(24 overall) || small squires size = 16x16 (64 in each big squire).
package wasys911992.blogger.example.thread;

import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import javax.imageio.ImageIO;

public class Square {
  /**
   * by github.com/911992
   * arashmd.blogspot.com
   */
  static public MySemaphore ms;
  public static ImageWriter iw;
  final static int maxThread=8;//maximum of thread at a time
  final static int blockSize=128;//big square block size in pixel
  final static int width=768,height=512;//these size should be dividable by blockSize
  final static String filePath="C:/output/rand2.png";//set the target image location
  public static void main(String[] args) throws InterruptedException, IOException {
    ms=new MySemaphore(maxThread); //initializing semaphore
    iw=new ImageWriter(width, height, blockSize); //initializing Image Writer
    for(int i=0;i<( (width/blockSize) * (height/blockSize) );i++){// for each big block [0]
      ms.acquire(); //get permission from semaphore [1]
      new Thread(new SquareMaker(i, blockSize)).start();//starts the new thread [2]
    }
    ms.join(); //waits until all threads finish their job
    iw.WriteToFile(filePath);// flush and save the image
    System.out.println("Done!");
  }

}
class SquareMaker implements Runnable{
  static int smallBlocks=8;//number of small block in each big square 8->(8*8)=64 small squares overall, should dividable by big square width
  int blockSize,blockNo;
  public SquareMaker(int blockNo,int blockSize){
    this.blockNo=blockNo;this.blockSize=blockSize;//setting block size(big square size) and the sequence number
  }
  @Override
  public void run() {
    int[] data=new int[blockSize*blockSize];//initializing the data array, it stores whole this big square colors
    int smallBlockSize=blockSize/SquareMaker.smallBlocks;//calculating how many pixels does a small square have?
    int index=0;int colorDiff=25;//maximum color difference value, would be more or less
    int baseColor=(int)(Math.random()*Integer.MIN_VALUE);//generate a random integer (color) value
    for(int i=0;i<(smallBlocks*smallBlocks);i++){//for each small square [3]
      //converting the randomized color into a 4 length array (alpha, red, green ,blue)
      byte[] color=ByteBuffer.allocate(4).putInt(baseColor).array();
      int diff=(int)(Math.random()*colorDiff)+1;//generates a randomized color differences
      int ndiff=(diff%2==0)?diff:diff*-1;//multiply with -1, needs for darker colors
      //checks the commented if-else block below, this is the short form of that.
      color[1]=((color[1]&0XFF)+colorDiff<255 && (color[1]&0XFF)-colorDiff>0)?
              (byte)((color[1]&0xFF)+ndiff):(((color[1]&0xFF)+colorDiff>255)?
              (byte)((color[1]&0xFF)-diff):(byte)((color[1]&0xFF)+diff));//[4]

      color[2]=((color[2]&0XFF)+colorDiff<255 && (color[2]&0XFF)-colorDiff>0)?
              (byte)((color[2]&0xFF)+ndiff):(((color[2]&0xFF)+colorDiff>255)?
              (byte)((color[2]&0xFF)-diff):(byte)((color[2]&0xFF)+diff));

      color[3]=((color[3]&0XFF)+colorDiff<255 && (color[3]&0XFF)-colorDiff>0)?
              (byte)((color[3]&0xFF)+ndiff):(((color[3]&0xFF)+colorDiff>255)?
              (byte)((color[3]&0xFF)-diff):(byte)((color[3]&0xFF)+diff));
        /*  if(((color[1]&0xFF)+colorDiff)>255){color[1]=(byte) ((color[1]&0xFF)-diff);}
          else if(((color[1]&0xFF)-colorDiff)<0){color[1]=(byte) ((color[1]&0xFF)+diff);}
          else{color[1]=(byte) ((color[1]&0xFF)+ndiff);}
          
          if(((color[2]&0xFF)+colorDiff)>255){color[2]=(byte) ((color[2]&0xFF)-diff);}
          else if(((color[2]&0xFF)-colorDiff)<0){color[2]=(byte) ((color[2]&0xFF)+diff);}
          else{color[2]=(byte) ((color[2]&0xFF)+ndiff);}
          
          
          if(((color[3]&0xFF)+colorDiff)>255){color[3]=(byte) ((color[3]&0xFF)-diff);}
          else if(((color[3]&0xFF)-colorDiff)<0){color[3]=(byte) ((color[3]&0xFF)+diff);}
          else{color[3]=(byte) ((color[3]&0xFF)+ndiff);}
          */
      for(int k=0;k<(smallBlockSize*smallBlockSize);k++){//for each pixel in the small squares [5]
        //finding the this small square pixels location in data array
        index+=(smallBlockSize*smallBlockSize)*Math.floor(i/smallBlocks)*smallBlocks;
        index+=Math.floor(k/smallBlockSize)*smallBlockSize*smallBlocks;
        index+=(k%smallBlockSize)+((i%smallBlocks)*smallBlockSize);
            data[index]=ByteBuffer.wrap(color).getInt();//wrap the color array to the integer(color) value
        index=0;
      }
    }
    Square.iw.writeSection(blockNo, data);//big square is done, pass the data array to write to the target image
    Square.ms.release();//tells semaphore this thread has completed

  }
}
class ImageWriter {
  BufferedImage out;
  int width,height,blockSize,blockWidth;
  public ImageWriter(int width, int height, int blockSize){
    this.width=width;this.height=height;this.blockSize=blockSize;this.blockWidth=this.width/this.blockSize;
    out=new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
  }
  int blockX,blockY;
  public synchronized void writeSection(int blockNo,int[] data){
    blockX=(blockNo%blockWidth)*blockSize;//find the location that this data(square) should get drawn
    blockY=(int)Math.floor(blockNo/blockWidth)*blockSize;
    System.out.println("Blockx: "+blockX+" blockY: "+blockY+" blockNo: "+blockNo+" array size: "+data.length);
    try{
      out.setRGB(blockX,blockY, blockSize, blockSize, data, 0, blockSize);//set the data to the target image
    }catch(Exception e){e.printStackTrace();}
  }
  public synchronized void WriteToFile(String path) throws IOException{
    out.flush();
    ImageIO.write(out, "png", new File(path));
  }
}
//find smeaphore tutorial here https://arashmd.blogspot.com/2013/06/java-threading.html
class MySemaphore{
  int maxThread,running;
  private Object joinWait=new Object();
  public MySemaphore(int maxThreads){
    this.maxThread=maxThreads;
  }
  public synchronized void acquire() throws InterruptedException{
    if(running==maxThread){this.wait();}
    running++;
  }
  public synchronized void release(){
    running--;
    if(running==0){synchronized(joinWait){try{joinWait.notify();}catch(Exception e){}}}
    try{
      this.notify();
    }catch(Exception e){}
  }
  public void join() throws InterruptedException{
    synchronized (joinWait) {
      if(running==0){return;}
      joinWait.wait();
    }
  }
}
We have a 768 x 512 pixel images, that we want to have 24 (128 x 128) big randomized color squares inside, and inside each big square there are 8 x 8 (64) small squares that each one is 16 x 16 pixel size.

Explanations

Usually image processes are run parallel, because there are too much small math and processes that would take a notable time if they get run linear, then threads would give a hand. Square class The main method calculates how many big squares does the image require, then in a for loop[0], gets the permission from semaphore[1] to run a thread (SquareMaker)[2].
After starting a thread, it(the thread) generates a randomized color (a color is represented by an int value RGBA).
Then for each small square inside [3], generates a darker/brighter color [4], and sets the color to the pixels that are belong to related small square [5].

The Disaster

I know you are still confused about the logic of using semaphore, I suggest you try to do these and see the differences.
  • Set the semaphore size to 1, it will be as same as single-thread application
  • Set the resolution to 4K (3840 x 2160), set the semaphore to a big value like 199000, and have a run

#6 Lively Squares

This is about a simple screen saver like application. It first goes for fullscreen mode, then next for n(levelCount) numbers creates smaller images that each one will passed to m(possbile width size squares) threads, so if there are 10 threads are working on a small image, it means there are 10 randomized color lively squares animating, something like the following.

Or like this
The image above doesn't show everything in detail unless you just run the code in your system and watch it live.
and how does this work? and how does it make squares?
For instance, if the screen resolution is 1280 x 768 and there is 9 column of squares is required (9 line of squares), then each square size will be (1280/9) 142 pixel x 142 pixels, 9 in a row and (768/142)5 in a column (40 lively square overall). check the example below.
As above diagram, the application finds there are 7 possible 250pixel size squares would be in a row, and 100 pixel free space in each left and right side, it's also same for top and down.
First application creates vertical image portion (yellow, and pink sections) that is 7 here, 7 (250x1200) size rectangles.
Then each vertical portion gets calculated how many possible 250x250(small square) would be inside this portion, and runs a separated thread for each one.
Finally each thread in a infinity loop begins to drawing own square, for example the very first small square here at point (0,100)(dark pink) and the next one in same portion at point(0,350).
BUT this is not end of the line, because the portions never get drawn directly to the screen.
Here a separated thread (drawing thread) clears and draws the portions every x ms, for example (1000/60) ms that causes 60 frame per second that sounds good, the lower refresh value, requires stronger system, but if you want to prevent of lagging you need at least 25~30 frames per second (1000/30).
Okay now let's have a look at the code, please read the comments they say everything while we will have explanation of course.
package wasys911992.blogger.example.thread;

import java.awt.*;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import java.awt.event.MouseWheelEvent;
import java.awt.event.MouseWheelListener;
import java.awt.image.BufferedImage;
import java.nio.ByteBuffer;

import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;

class Parameter{//Parameter class for passing required data to threads and between classes
  public static final Color backColor=Color.black;//set the background color, white and black are cool enough :D
  public static int minDrawTime=33;//the minimum time square drawer should wait after each drawing process
  public Graphics2D g;//graphic that associated to each vertical portion, for drawing easier
  public int width,height;//indicated the width and height of eahc portion
  Parameter( Graphics2D g, int width, int height) {
    this.g = g;
    this.width = width;
    this.height = height;
  }
}
/**
 * by github.com/911992
 * arashmd.blogspot.com
 */
public class Matrix implements Runnable {//Runnable implemented, this is the drawing thread
  public static void main(String[] args) throws Exception {
    new Matrix();
  }
  JLabel lab; //Label for showing the target image
  //just some string at the top of the target image, would be your name :D
  String message="by github.com/911992(arashmd.blogspot.com) , wheel the mouse for changing the drawing speed, click to exit";
  int levelCount=24;//number of vertical portion, change and see the effects (big ones doesn't make sense)
  int capSpace;//the space reminded (unusable) at the right and left sides 
  BufferedImage img;//the target image that shows the target
  Graphics2D g;//the graphic associated the the img variable(target image)
  
  //number of times(frame) that drawing thread needs to refresh the target image in a second
  //the higher value will effects(improve) the animation smoothness and would cause the vertical-synchronization also needs more power
  //the lower value needs less power, decrease the animation smoothness and would cause the glitching
  int fps=60;//do not set less than 30 and and high values
  
  
  BufferedImage[] images;//arrays of portion images
  int screenWidth,screenHeight;//the width and height of the screen
  Matrix() throws Exception{
    GraphicsDevice vc;
    JFrame f=new JFrame("Matrix by Arash");
    GraphicsEnvironment ge=GraphicsEnvironment.getLocalGraphicsEnvironment();
    Toolkit tk=Toolkit.getDefaultToolkit();
    //hide the cursor
    f.setCursor(f.getToolkit().createCustomCursor(new BufferedImage(1, 1, BufferedImage.TYPE_INT_ARGB), new Point(0, 0), "null"));
    screenWidth=tk.getScreenSize().width;//get the screen size
    screenHeight= tk.getScreenSize().height;
    img=new BufferedImage(screenWidth,screenHeight, BufferedImage.TYPE_INT_ARGB);//init the target image as screen size
    lab=new JLabel(new ImageIcon(img), JLabel.CENTER);//init label and pass the target image to it as showing parameter
    f.getContentPane().add(lab);
    g=img.createGraphics();
    vc=ge.getDefaultScreenDevice();
    f.setUndecorated(true);
    f.setResizable(false);
    vc.setFullScreenWindow(f);//set the application as full screen
    lab.addMouseListener(new MouseListener() {
      @Override
      public void mouseReleased(MouseEvent arg0) {}
      @Override
      public void mousePressed(MouseEvent arg0) {}
      @Override
      public void mouseExited(MouseEvent arg0) {}
      @Override
      public void mouseEntered(MouseEvent arg0) {}
      @Override
      public void mouseClicked(MouseEvent arg0) {System.exit(0);}//exit the by a click
    });
    //listening for mouse wheel for decreasing and increasing the fade-out/fade-in operation
    lab.addMouseWheelListener(new MouseWheelListener() {
      @Override
      public void mouseWheelMoved(MouseWheelEvent arg0) {
        if(arg0.getWheelRotation()==-1){
        if(Parameter.minDrawTime>30){Parameter.minDrawTime-=10;}}
      else{if(Parameter.minDrawTime<60){Parameter.minDrawTime+=10;}}}
    });
    images=new BufferedImage[levelCount];//init the portion images array
  //calculate the remind(unusable) space at  right/left sides in target image
    capSpace=(int)(tk.getScreenSize().getWidth()%levelCount)/2;
  //calculate how many pixel is required for each portion image
    int width= (int) Math.floor(tk.getScreenSize().getWidth()/levelCount);
    for(int i=0;i<levelCount;i++){//in as for loop init portion image array images
       images[i]=new BufferedImage(width,tk.getScreenSize().height,BufferedImage.TYPE_4BYTE_ABGR);
       Parameter p=new Parameter(images[i].createGraphics(),width,tk.getScreenSize().height);
       new Thread(new  VerticalPortion(p)).start();//starts a portion as a thread
    } 
    Font font=new Font("courier new",Font.BOLD,16);//the font for showing the message at the top screen
    g.setFont(font);
    new Thread(this).start();//starts this thread(drawing thread)
  }


@Override
public void run() {//drawing thread start
  while(true){
    g.setColor(Parameter.backColor);//set the color as it should be
    g.fillRect(0, 0, screenWidth, screenHeight);//purge the screen
    for(int i=0;i<images.length;i++){//draw each portion to the target img
      g.drawImage(images[i],(images[i].getWidth()*i)+((i==0)?capSpace:0),0,lab);
    }
    g.setColor(Color.gray);//set the color to draw the message, could be anything
    g.drawString(message,20, 20);
    lab.repaint();//force the label to refresh (refresh the image in fact)
    try {
      Thread.sleep(1000/fps);//wait for a while
    } catch (InterruptedException e) {}
  }
}//drawing thread end
}
class VerticalPortion implements Runnable{
  Parameter p;
  Thread[] trds; 
  int verticalSpaceCap;//remind (unusable) space in top and bottom of the screen
  public VerticalPortion(Parameter p){
    this.p=p;
    trds=new Thread[(int) Math.floor(p.height/p.width)];//calculate how many possible square would this portion have
    verticalSpaceCap=p.height%p.width;//calculate the unusable space
    p.g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, 
        RenderingHints.VALUE_ANTIALIAS_ON);//enable antialias, it draws components smoother
  }
  @Override
  public void run() {
    for(int i=0;i<trds.length;i++){
        Parameter px=new Parameter(p.g,p.width,p.height);
        trds[i]=new Thread(new SmallSquare(px,i*p.width+(verticalSpaceCap/2)));//one thread for one small square handler 
      trds[i].start();//start the thread
    }
  }
}
class SmallSquare implements Runnable{
    Parameter p;
    //two 4 length array that contains the RGBA of both background color and color square should be drawn by
    byte[] color,hcolor=new byte[4];
    int topSpaceCap;//indicates how many pixel from top should this thread skip
int margin; //spaces should be skip at each edge of square
int j;//index for saving the last alpha value
int waitTime;//time this thread should wait after each drawing 
int shapeSize;//the target shape size
int increaseWidth;//the increment size for each square layer
int colorDiff=10;//the value that effects each iteration color, could be more(more differences) or less
public SmallSquare(Parameter p,int topSpaceCap) {
  this.p=p;
  this.topSpaceCap=topSpaceCap;
  margin=(int) (p.width*0.05);
  //split the randomized color into a 4 byte array as ARGB
  this.color=ByteBuffer.allocate(4).putInt((int)(Math.random()*Integer.MIN_VALUE)).array();
  hcolor[0]=(byte)0;
  hcolor[1]=(byte)Parameter.backColor.getRed();
  hcolor[2]=(byte)Parameter.backColor.getGreen();
  hcolor[3]=(byte)Parameter.backColor.getBlue();
  //Initialize the target full-transparent square size 
  shapeSize=(p.width)-(margin*2);
  //initialize how many pixel should increase for the next layers
  increaseWidth=(int) ((shapeSize-(shapeSize*0.6))/10);
}

@Override
public void run() {
  while(true){
  waitTime=(int)(Math.random()*Parameter.minDrawTime*3);//get a random wait time in each iteration
  if(waitTime<Parameter.minDrawTime){waitTime=Parameter.minDrawTime;}
    int diff=(int)(Math.random()*colorDiff)+1;
    int ndiff=(diff%2==0)?diff:diff*-1;
    color[1]=((color[1]&0XFF)+colorDiff<255 && (color[1]&0XFF)-colorDiff>0)?
        (byte)((color[1]&0xFF)+ndiff):(((color[1]&0xFF)+colorDiff>255)?
            (byte)((color[1]&0xFF)-diff):(byte)((color[1]&0xFF)+diff));
        
    color[2]=((color[2]&0XFF)+colorDiff<255 && (color[2]&0XFF)-colorDiff>0)?
        (byte)((color[2]&0xFF)+ndiff):(((color[2]&0xFF)+colorDiff>255)?
            (byte)((color[2]&0xFF)-diff):(byte)((color[2]&0xFF)+diff));
        
    color[3]=((color[3]&0XFF)+colorDiff<255 && (color[3]&0XFF)-colorDiff>0)?
        (byte)((color[3]&0xFF)+ndiff):(((color[3]&0xFF)+colorDiff>255)?
            (byte)((color[3]&0xFF)-diff):(byte)((color[3]&0xFF)+diff));
        
        //fade-in method 0, comment the method 1 if you wan to use this
      /*for(int i=0;i<10;i++){
        if(i==9){color[0]=(byte) 255;}
        else{color[0]=(byte) (i*10);}
        synchronized (p.g){
          if(i==9){p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,color[0]&0xFF).brighter());}
          else{p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,color[0]&0xFF));}
          p.g.fillRoundRect(margin+(i*increaseWidth),margin+(i*increaseWidth)+this.topSpaceCap,
                shapeSize-(i*increaseWidth*2),shapeSize-(i*increaseWidth*2),j,j);
        }
        try {Thread.sleep(waitTime);} catch (InterruptedException e) {}
      }*/  //fade-in method 0
        
        
        //fade in method 1
    for(int i=10;i>0;i--){
      color[0]=(byte) (i*5);//sets the alpha for each layer of the square
      
      /*this synchronized block is required, because in a portion, there are(could be) more than 1 thread.
       *so because the portion image is shared belong to every thread, each thread need to lock the graphic object
       *and draw it's image, if there is no synchronized method, this is possible that thread 1 draw its line by 
       *the color that thread 2 has been specified, just omit the synchronized block and see the difference!
       */
      synchronized (p.g){//[0]
        //drawing brighter color for the center square
        if(i==10){p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,255).brighter().brighter().brighter());}
        else{p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,color[0]&0xFF));}
        p.g.fillRoundRect(margin+(i*increaseWidth),margin+(i*increaseWidth)+this.topSpaceCap,
            shapeSize-(i*increaseWidth*2),shapeSize-(i*increaseWidth*2),(10-i)*2,(10-i)*2);
      }
      try {Thread.sleep(waitTime);} catch (InterruptedException e) {}      
    }//fade-in method 1
    
    //fade-out
      for(int i=0;i<50;i++){
      hcolor[0]=(byte) (i*5.1);
      synchronized (p.g){
        p.g.setColor(new Color(hcolor[1]&0xFF,hcolor[2]&0xFF,hcolor[3]&0xFF, hcolor[0]&0xFF));
        p.g.fillRect(margin,margin+this.topSpaceCap,shapeSize,shapeSize);
      }
        try {Thread.sleep(waitTime);} catch (InterruptedException e) {}
    }//fade-out
      
   }
  }
}
We didn't use either semaphore or thread pool there, because this example requires kinda always-running threads.
Well this is a very simple example, I don't go for explaining the math processes and operations because we are here for threads beside the math subjects are not to much complex (never fear of lengthy codes), but if you think there is some explain is needed for math formulas and other subject, feel free to tell me, of course I will explain.

Explination

Drawing Thread
@Override
public void run() {//drawing thread start
  while(true){
    g.setColor(Parameter.backColor);//set the color as it should be
    g.fillRect(0, 0, screenWidth, screenHeight);//purge the screen
    for(int i=0;i < images.length ; i++){//draw each portion to the target img
      g.drawImage(images[i],(images[i].getWidth()*i)+((i==0)?capSpace:0),0,lab);
    }
    g.setColor(Color.gray);//set the color to draw the message, could be anything
    g.drawString(message,20, 20);
    lab.repaint();//force the label to refresh (refresh the image in fact)
    try {
      Thread.sleep(1000/fps);//wait for a while
    } catch (InterruptedException e) {}
  }
}//drawing thread end
This thread like other threads has got a while(true) loop, it means it have to do the business forever unless there is a halt signal. In each iteration this thread purges(clears) the whole screen by drawing(filling) a screen-sized rectangle, then by a for loop draws the each portion.
At the end it needs to wait for a while(small time), this sleeping time ensures thread that user seeing the result with this time, as we are human, we could see maximum around(near) 30 frame(picture) in a second, but if there is 60 frame in second, we just see it smoother(no actual 60 frames).
Small Square Drawing Thread
The servants, threads that draw squares, each thread draws(animates) a(its) square into a certain position of passed image(portion), so this is possible that 5 threads are drawing 5 squares in a shared image, so because each square has different color then each thread needs to lock the image for drawing, then release the image in order to let another thread acquires the locks and does the same.
IF there is not any lock, then it's possible thread 1 sets the color to white and starts to drawing, while thread 1 is drawing, thread 2 sets the color to black, so it causes that square in thread 1 has two(or more) color inside.
@Override
public void run() {
  while(true){
  waitTime=(int)(Math.random()*Parameter.minDrawTime*3);//get a random wait time in each iteration
  if(waitTime<Parameter.minDrawTime){waitTime=Parameter.minDrawTime;}
    int diff=(int)(Math.random()*colorDiff)+1;
    int ndiff=(diff%2==0)?diff:diff*-1;
    color[1]=((color[1]&0XFF)+colorDiff<255 && (color[1]&0XFF)-colorDiff>0)?
        (byte)((color[1]&0xFF)+ndiff):(((color[1]&0xFF)+colorDiff>255)?
            (byte)((color[1]&0xFF)-diff):(byte)((color[1]&0xFF)+diff));
        
    color[2]=((color[2]&0XFF)+colorDiff<255 && (color[2]&0XFF)-colorDiff>0)?
        (byte)((color[2]&0xFF)+ndiff):(((color[2]&0xFF)+colorDiff>255)?
            (byte)((color[2]&0xFF)-diff):(byte)((color[2]&0xFF)+diff));
        
    color[3]=((color[3]&0XFF)+colorDiff<255 && (color[3]&0XFF)-colorDiff>0)?
        (byte)((color[3]&0xFF)+ndiff):(((color[3]&0xFF)+colorDiff>255)?
            (byte)((color[3]&0xFF)-diff):(byte)((color[3]&0xFF)+diff));
      
        //fade in method 1
    for(int i=10;i > 0;i--){
      color[0]=(byte) (i*5);//sets the alpha for each layer of the square
      synchronized (p.g){//[0]
        //drawing brighter color for the center square
        if(i==10){p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,255).brighter().brighter().brighter());}
        else{p.g.setColor(new Color(color[1]&0xFF,color[2]&0xFF,color[3]&0xFF,color[0]&0xFF));}
        p.g.fillRoundRect(margin+(i*increaseWidth),margin+(i*increaseWidth)+this.topSpaceCap,
            shapeSize-(i*increaseWidth*2),shapeSize-(i*increaseWidth*2),(10-i)*2,(10-i)*2);
      }
      try {Thread.sleep(waitTime);} catch (InterruptedException e) {}      
    }//fade-in method 1
    
    //fade-out
      for(int i=0;i < 50;i++){
      hcolor[0]=(byte) (i*5.1);
      synchronized (p.g){
        p.g.setColor(new Color(hcolor[1]&0xFF,hcolor[2]&0xFF,hcolor[3]&0xFF, hcolor[0]&0xFF));
        p.g.fillRect(margin,margin+this.topSpaceCap,shapeSize,shapeSize);
      }
        try {Thread.sleep(waitTime);} catch (InterruptedException e) {}
    }//fade-out
      
   }
  }
Find the [0](synchronized block) in above code, just comment this block and rerun the application, you will see the inconsistency in square colors that is very bad (it's awful), we could design and write in such a way that there is no any dependency for locking. Each thread could creates and works with its own image.

Exercises

and now it's your time, copy-pasting is enough, if you really want to learn parallel programming, you need to do real samples too yourself, so prepare a coffee and get ready for codingz! Ha ha
Okay the following examples are really easy, they may need to much code, but they are simple just same as above examples, and you would just contact me for verifing your work (if I can understand).
  • A horse race game: this is very old and simple chance game, you know enough, but just a mention, it's about 5-7 horses that at the start the game each horse(thread) generates a random number (maybe between 14-16), and starts to reach the end line, while you bet on a horse that would win.
  • Counting numbers: a simple screen-like application (like lively squares), there some counting numbers in different sizes(one could get 20% of whole page, randomized) that each one are increasing, randomely each number has a increasing count.
  • Cursor Enemies: (Zombies needs cursor) it's kind of fun gui application, okay there is full screen page with 5 slow-moving objects(a square, rectangle, something) that those guys find(move through) the cursor current position to reach the cursor, user has to run a way from objects. If any objects reaches the cursor position user will lost.
Hope this article was good and affective for you dear reader, any command and feedback is so much appreciated. Happy coding :D