Wednesday, October 13, 2010

How to find the exact instance with session id in Weblogic cluster using curl or lynx

Suppose we have 90 weblogic app server instances running behind the load balancers and AKAMAI.
If a user faces an error/exception in one scenario. If we have the user session id (For example: xGGwM2Hpq0lhCr!-1529204333)
Note: For each server we will have jvm id. The part after delimeter !- is jvmid or serverid which will not be changed until server restarts.

To find the exact server so that we directly jump to exact logs. Instead of going through all 90 instances.
Using lynx - findhost.sh
#!/bin/bash
# Iterate through each host
for i in {20..109};
do
  # get jvmid or server id the part after delimeter !- in session id
  jvmid=$(lynx -dump -head http://host$i:7001/business/ | grep -i session  | sed 's/^.*!\|^.*-\|;.*//g')
  echo "p2pre1c$i : $jvmid"
done
or
Using curl - findhost.sh
#!/bin/bash
# Iterate through each host
for i in {20..109};
do
  # get jvmid or server id the part after delimeter !- in session id
  jvmid=$(curl --head --silent http://host$i:7001/business/ | grep -i session  | sed 's/^.*!\|^.*-\|;.*//g')
  echo "p2pre1c$i : $jvmid"
done

Explanation:
curl --head  and lynx -dump -head
will dump the headers which will have session id
For example:
# lynx -dump -head http://host20:7001/business/
HTTP/1.1 200 OK
Cache-Control: no-cache="set-cookie"
Connection: close
Date: Thu, 14 Oct 2010 00:20:18 GMT
Content-Type: text/html; charset=UTF-8
X-ATG-Version: version=QVRHUGxhdGZvcm0vOS4xIFsgRFBTTGljZW5zZS8wIEIyQkxpY2Vuc2Uv
MCAgXQ==
Set-Cookie: DSESSIONID=FnHsM2MCR3ZpJj!233685404; path=/
Set-Cookie: browserid=A001287015618072FnHsM2MCR3ZpJj!233685404!1287015618064; d
omain=.wireless.att.com; expires=Friday, 14-Oct-2011 00:20:18 GMT; path=/
Set-Cookie: cust_type=b2b; domain=.wireless.att.com; expires=Thursday, 21-Oct-2
010 00:20:18 GMT; path=/
X-Powered-By: Servlet/2.5 JSP/2.1

$ ./findhost.sh
host20 : 233685404
host21 : 2060847712
host22 : 447882153
host23 : 1405278242
host24 : 2099132956
...

If user sessionid is FnHsM2MCR3ZpJj!233685404
the instance serving the above user session is host20.

Thursday, October 7, 2010

Few commands to know about solaris box

Mostly Solaris is used as production server since it is more robust.

To find total hard disk space
$ iostat -E
sd1       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE  Product: ST373207LSUN72G  Revision: 045A Serial No: 0543330VQB
Size: 73.40GB <73400057856 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd2       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: FUJITSU  Product: MAW3073NCSUN72G  Revision: 1703 Serial No: 0809B0R45J
Size: 73.40GB <73400057856 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd3       Soft Errors: 1 Hard Errors: 0 Transport Errors: 0
Vendor: TOSHIBA  Product: ODD-DVD SD-C2732 Revision: 1055 Serial No: 
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
sd4       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: FUJITSU  Product: MAW3073NCSUN72G  Revision: 1703 Serial No: 0751B0PJ01
Size: 73.40GB <73400057856 bytes>

To find free hard disk space available
$ df -kh
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0         7.9G   3.1G   4.7G    40%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.0G   1.3M   5.0G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object

To find how much disk space used
$ du -sh /
  34G  

To find how much disk space used by a directory
$ du -sh /tmp
 869M   /tmp

To find Total RAM
$ /usr/sbin/prtconf -v | head -2
System Configuration:  Sun Microsystems  sun4u
Memory size: 8192 Megabytes

$ /usr/sbin/prtdiag -v | head -3
System Configuration: Sun Microsystems  sun4u Sun Fire V440
System clock frequency: 177 MHZ
Memory size: 8GB       

To monitor RAM usage by each process
$ top
load averages:  0.03,  0.03,  0.02;                    up 41+21:17:50                                               13:46:14
73 processes: 72 sleeping, 1 on cpu
CPU states: 99.0% idle,  0.4% user,  0.6% kernel,  0.0% iowait,  0.0% swap
Memory: 8192M phys mem, 2906M free mem, 8193M total swap, 8193M free swap

   PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
   923 wlsadmin 166  59    0 3290M 1305M sleep   37:47  0.13% java
 28115 wlsadmin 197  59    0 3290M 1344M sleep   63:36  0.12% java
 28164 wlsadmin 142  59    0 3010M  926M sleep   31:51  0.12% java

To know all environment variables that are set
$ env
MANPATH=/usr/share/man:/usr/local/man:/usr/dt/share/man:/usr/openwin/share/man:/usr/local/share/man:/opt/VRTS/man
TERM=vt100
SHELL=/bin/bash
SSH_CLIENT=141.204.63.62 1539 22
SSH_TTY=/dev/pts/2
USER=m64711
MAIL=/usr/mail/m64711
PATH=/usr/bin:/usr/local/bin:/usr/openwin/bin
PWD=/home/m64711
EDITOR=vi
TZ=US/Central
PS1=${LOGNAME}@usppre16:${PWD}$
SHLVL=1
HOME=/home/m64711
LOGNAME=m64711
SSH_CONNECTION=141.204.63.62 1539 135.214.192.188 22
_=/usr/bin/env

To know if the solaris running in 32 or 64 bit mode
The isainfo utility can  answer are  whether  64-bit  applications are supported, or whether
the running kernel uses 32-bit or 64-bit device drivers.
$ isainfo -v
64-bit sparcv9 applications
        vis2 vis
32-bit sparc applications
        vis2 vis v8plus div32 mul32

if you get only 32-bit then it is running in 32 bit mode
if you get both 64-bit and 32-bit then it is running in 64 bit mode

The uname utility prints information about the current  system on the standard output.
$ uname -X
System = SunOS
Node = bsdevprem01
Release = 5.10
KernelID = Generic_127111-08
Machine = sun4u
BusType = <unknown>
Serial = <unknown>
Users = <unknown>
OEM# = 0
Origin# = 1
NumCPU = 2

To know the installed JVM is capable of running 64-bit version
$ java -d64 -version
java version "1.5.0_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_15-b04, mixed mode)

if not 64 bit capable you will get some message like

This Java instance does not support a 64-bit JVM. Please install the desired version.

To know how long the serer is up
$ uptime
  5:15pm  up 78 day(s),  8:39,  3 users,  load average: 6.77, 6.75, 7.46

To know about active processes running
$ ps -ef | grep -i startweblogic
wlsadmin 28145 28140   0   Oct 01 ?           0:00 /bin/sh /sites/WLS/DOMAINS/PREMIERQA1-CASTAGING/bin/startWebLogic.sh
wlsadmin 28096 28091   0   Oct 01 ?           0:00 /bin/sh /sites/WLS/DOMAINS/PREMIERQA1-CAMAIN1/bin/startWebLogic.sh
wlsadmin 28140     1   0   Oct 01 ?           0:00 /bin/sh /sites/WLS/DOMAINS/PREMIERQA1-CASTAGING/startWebLogic.sh

-e list every process running now
-f list full information

For more info man ps

Using SVN and sqlplus

sqlplus example
devdb.sh
sqlplus username/password@'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname)(PORT=hostport)))(CONNECT_DATA=(SID=schemaname)))'

SVN Example
mysvn.sh
svn checkout "svn://hostname/B2B/branches/trunk/Module@HEAD" -r HEAD --depth infinity "~/workspace/BUILD" --username "xyz" --password "password"

Using curl to send/receive data from/to server

curl is a client which sends and receives documents from server without much GUI interaction.

In ATG, we have findclass.jhtml, which will let us know from where the class is refereed from.
for example: http://localhost:7001/dyn/dyn/findclass.jhtml


If you view page source of the findclass.jhtml,
you will see form inside findclass.jhtml has get method which is expecting className.

So by using below command you will get the result
curl -silent -d "className=atg.adapter.gsa.GSARepository" http://localhost:7001/dyn/dyn/findclass.jhtml | grep zip
will give the path from where the class is reffered from.
Explanation:
use man curl for more info
curl is a tool to transfer data from or to a server.
-d : Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button

For example: If you want to set loggingdebug to true of any repository  ex: /atg/userprofiling/ProfileAdapterRepository

The form method will be like below
<form method="POST" action="/dyn/admin/nucleus/atg/userprofiling/ProfileAdapterRepository/">
<input type="hidden" value="loggingDebug" name="propertyName">
<input type="radio" checked="" value="true" name="newValue">true<br>
<input type="radio" value="false" name="newValue">false
<p><input type="submit" value="Change Value" name="change">
</p><p>(Note that values changed here are not written permanently to the properties files)
</p><p>
</p><h1><a name="properties"></a>Properties</h1>
<table border="1">
<tbody><tr><th>Name</th><th>Value</th><th>Type</th></tr>
<tr class="even"><td><a href="/dyn/admin/nucleus/atg/userprofiling/ProfileAdapterRepository/?propertyName=loggingDebug.class">class</a></td><td>Class <a href="/dyn/admin/nucleus/atg/dynamo/docs/apidoc/java/lang/Boolean.html">java.lang.Boolean</a></td><td>java.lang.Class</td></tr>
</tbody></table>
</form>
So
curl -u admin:admin -silent -d "newValue=true&change=Change Value" http://bsdevprem01:7001/dyn/admin/nucleus/atg/userprofiling/ProfileAdapterRepository/?propertyName=loggingDebug > /dev/null
-u is to send username:password for admin UI
-d is to post the required form data
> /dev/null redirecting output to null
In more formal way

onDebug.sh
if [ -n "$1" ]; then
curl -u admin:admin -d "newValue=$4&pchange=Change%20Value" http://$1/dyn/admin/nucleus$2/?propertyName=$3
else
echo "Usage: ./curlexample localhost:7001 /com/att/wireless/recommendation/PopularDevices loggingDebug true"
fi
----------------------------------------------------------------------------------

Wednesday, October 6, 2010

Using "less" to view server logs

To view logs and search for any pattern, less would be great tool to use.
If logs are archived using pid or date (refer post http://lynuxkid.blogspot.com/2010/10/to-run-process-at-background-and.html for more info)

use "ls -ltr *pattern*" to get latest logs (if many instances logs are in same directory)
$ ls -ltr *PQA1logs*
-rw-r--r--   1 wlsadmin atg      16455232 Sep 28 17:15 PQA1logs.txt.4728104734
-rw-r--r--   1 wlsadmin atg      17439042 Sep 29 08:44 PQA1logs.txt.1628171655
-rw-r--r--   1 wlsadmin atg      20182090 Sep 29 16:31 PQA1logs.txt.4629084624
-rw-r--r--   1 wlsadmin atg      26107020 Oct  1 15:04 PQA1logs.txt.3329163306
-rw-r--r--   1 wlsadmin atg      41899784 Oct  6 13:31 PQA1logs.txt.1501171516

then use less -i (i is to ignore case while searching)
$ less -i PQA1logs.txt.1501171516
Keybindings for less will be similar to Vi (refer post http://lynuxkid.blogspot.com/2010/10/vi-commands.html for Vi commands)

Traversing file
G - To go to end of the file
g - to go to start of the file
j - down one line
k - up one line
space - go forward one page
b - go back one page
F - to refresh page to get latest logs (similar to tail)
Ctrl+c to cancel any running operation for example - to cancel refresh of page.
:100 jump to 100th line

Forward search
/pattern - to search for pattern
n - to search next match
N - to search previous match

Backward search
?pattern - to search for pattern
n - to search previous match
N - to search next match

For more info
h - Summary of less commands.

For gunzip files with .gz - use gzless with above options.
Use gzgrep to grep in .gz file.

Monday, October 4, 2010

Using rsync

To transfer files from one location (localhost or remote) to localhost or
to transfer files from localhost to remote
rsync -tracdv --delete username@hostname:/sites/scripts/env2 /sites/env2
-t - preserve timestamp
-r - recurse into directories
-a - archive mode
-c - skip based on checksum, not mod-time & size
-d - transfer directories without recursing
-v - verbose mode
-- delete - delete files in destination if source has deleted them.

for more info
man rsync
rsync -h

To find/replace a string in all current directory files and sub directory files

grep -i test $(find . -name "*.txt")
sample output :
$ grep -i test $(find . -name "*.txt")
./apiclasses_all.txt:atg.commerce.order.processor.ProcValidateStoreCredit

you can also try with
$ find . -name "*.txt" -exec grep -i test {} \; -print
atg.commerce.order.processor.ProcValidateStoreCredit
./apiclasses_all.txt

In some environments grep with recursive will work
$ grep -r -i raj *.txt
test1.txt:raj
text2.txt:raj

If there are many instances running on same machine and logs are redirecting to same directory
for example:

$ ls -ltr
total 1779040
-rw-r--r--   1 sr8879   sysadmin    6243 Aug  5 10:42 jdbc.rar
drwxr-xr-x   2 root     root         178 Aug 26 16:29 hsperfdata_root
drwx------   2 root     root         117 Aug 26 16:30 vx.004203.035257.110400.1642
drwx------   2 root     root         117 Aug 26 16:30 vx.106663.005702.043400.1753
drwx------   2 root     root         117 Aug 26 16:30 vx.113510.132240.132400.1883
-rw-------   1 root     other        133 Aug 26 16:30 sh22020
-rw-------   1 root     other        430 Aug 26 16:30 croutAAA4Aaisc
-rw-r--r--   1 wlsadmin atg       203422 Aug 27 05:07 dynolog.PREMIERQA1CAAPP-20100827050632
-rw-r--r--   1 wlsadmin atg       352234 Aug 27 05:14 PQACAMAIN1logs.txt.3326163321
-rw-r--r--   1 wlsadmin atg       179579 Aug 27 05:14 PQACASTAGlogs.txt.3326163343
-rw-r--r--   1 wlsadmin atg       892162 Aug 27 16:18 PQACAMAIN1logs.txt.1427051427
-rw-r--r--   1 wlsadmin atg       178808 Aug 27 16:19 PQACASTAGlogs.txt.1427051449
-rw-r--r--   1 wlsadmin atg       172779 Aug 27 16:52 dynolog.PREMIERQA1CAAPP-20100827165130

you want to search for error containing "repository does not have SELECT permission" only in Staging logs
PQACASTAG* logs belongs to Staging instance
So inorder to search those logs

To get only PQACASTAG* logs
by using grep and sed
$ ls -ltr | grep -i 'PQACASTAG*' | sed 's/^.*PQA/PQA/g'
PQACASTAGlogs.txt.3326163343
PQACASTAGlogs.txt.1427051449
PQACASTAGlogs.txt.5927165947

or  by using awk and grep
$ ls -ltr | awk '{print $9}' | grep -i 'PQACASTAG*'
PQACASTAGlogs.txt.3326163343
PQACASTAGlogs.txt.1427051449
PQACASTAGlogs.txt.5927165947

Now we need to search only those logs
$ grep -i error.*select.*permission $(ls -ltr | awk '{print $9}' | grep -i 'PQACASTAG*')
PQACASTAGlogs.txt.3326163343:**** Error Thu Aug 26 16:51:57 PDT 2010    1282866717694   /com/att/wireless/reporting/ReportingRepository      The table: "V_ONLINE_ORDER_INFO" does not appear to be defined correctly in the database. Either the table does not exist, the repository does not have SELECT permission on it, or the definition of the table is incompatible with the template in use.

PQACASTAGlogs.txt.3326163343:**** Error Thu Aug 26 16:51:57 PDT 2010    1282866717701   /com/att/wireless/reporting/ReportingRepository      The table: "V_ONLINE_ORDER_STATUS_INFO" does not appear to be defined correctly in the database. Either the table does not exist, the repository does not have SELECT permission on it, or the definition of the table is incompatible wit
h the template in use.


To replace all "text" in all ".txt" files in current directory
find . -name "*.txt" -exec sed -i 's/text/raj/g' {} \;
or
sed -i 's/raj/text/g' $(find . -name "*.txt")
  
Displaying all html's which contains a word as links in browser (more like a local search engine)
grep -il $1 $(find /cygdrive/c/ATGPlatform91docs/ -name "*.htm[l]" -type f) | sed 's/\(.*\)/\1<\/a>
/g' > search.html; links search.html
to use
./satg Repository
which will display results in links browser with all html's as links which contains"Repository".
You can either use links or lynx or firefox to view search.html

To run process at background and redirecting the output/logs to a file.

For Example : To run startWeblogic.sh at background and redirect output/logs to CA1.log
and how to archive the logs.

startwls.sh
nohup /sites/WLS/DOMAINS/CA1/bin/startWebLogic.sh >/tmp/CA1_$$.log 2>&1 &
In above script, we are redirecting the logs to CA1_$pid at server start up.
Explanation:
/sites/WLS/DOMAINS/CA1/bin/startWebLogic.sh is the command used to start weblogic
To run the above command background we need use nohup
nohup /sites/WLS/DOMAINS/CA1/bin/startWebLogic.sh &
The above command makes the startWeblogic.sh command to run background and the logs will be redirected to nohup.out
But if we use above command, we cant start run another process backgroud with nohup because nohup.out is used by another process

So for this purpose
We use " > /tmp/CA1.log 2>&1".
> is used to redirect logs
&1 is our /tmp/CA1.log and 2>&1 to redirect standard error to stdout (i.e 1).

So the command is -
nohup /sites/WLS/DOMAINS/CA1/bin/startWebLogic.sh >/tmp/CA1.log 2>&1 &

But by command, if server is restarted previous logs are gone.

To archive previous logs, we can use pid (process id). Inorder to achieve that we can append process id variable ($$)
So total command is
nohup /sites/WLS/DOMAINS/CA1/bin/startWebLogic.sh >/tmp/CA1_$$.log 2>&1 &

or else to be precise u can also append date with time
temp_$(date +%m%d%y)_$(date +%H%M%S).log

 if we have logs with pid attached, how to view the latest/current log.
 For example:
$ls -ltr *CAMAIN1*
-rw-r--r--   1 wlsadmin atg       490353 Sep 29 08:50 PQACAMAIN1logs.txt.2128172119
-rw-r--r--   1 wlsadmin atg       371140 Sep 29 16:37 PQACAMAIN1logs.txt.5029085048
-rw-r--r--   1 wlsadmin atg       537022 Oct  1 15:13 PQACAMAIN1logs.txt.3729163730
-rw-r--r--   1 wlsadmin atg       387844 Oct  4 13:41 PQACAMAIN1logs.txt.2301172351

showlogs.sh
less -i /tmp/$(ls -ltr /tmp/*CAMAIN1* | sed s/^.*PQA/PQA/g | tail -1)
or
by using awk
less -i $(ls -ltr /tmp/*CAMAIN1* | tail -1 | awk '{print $9}')
By above script, we will "less" the latest CA1*.log.

Explanation:
ls -ltr /tmp/*CAMAIN1* - list all the logs with CAMAIN1 (listing from least edited to recently edited).
sed s/^.*PQA/PQA/g - used sed to get exact filename.
tail -1 - to list the last line
awk '{print $9}' - print 9th column of the row

Using pipe '|" run all the above commands parllely
Once u got the name of the file open using less command. (-i is to ignore case while searching).

How to List all/specified files in all jar's recursively in a directory

To list contents of a all jar files in current directory in below format
./wlserver_10.0/server/lib/api.jar : javax/net/ssl/impl/SSLSocketFactoryImpl.class

expandjars.sh
#!/bin/bash
JAR_LIST=$(find . -type f -name "*.jar") # find all files with .jar as extension.
for file in ${JAR_LIST}
do
     CLASS_LIST=$(jar -tvf ${file} | grep -i '.properties\|.java\|.xml\|.class' | sed 's/^.* //g')
     for class in ${CLASS_LIST}
    do
          echo "${file} : ${class}"
    done
done

If you are using Cygwin. After copying the code use dos2unix to avoid " syntax error near unexpected token"
dos2unix expandjars.sh

Explanation:
find . -type f -name "*.jar" - will find all the jar files in the current directory.
Store the output of above command in JAR_LIST.
Using "for" iterate for every entry (jar) in JAR_LIST
jar -tvf will unjar. Using grep find the files which you need.
echo them to stdout.

./expandjars.sh > classes.txt
once u get classes.txt
use grep to find the respective class like 'grep -i classname classes.txt'

Vi Commands

****************************************
/* HOW TO OPEN FILE THRU VI EDITOR*/
***************************************
:e filename/vi filename to open file in vi
:view filename to only view the file. restricted to write
:enew to edit new file
:new to edit new file in H split
:vnew to edit new file in V split
:tabe to edit new file in new tab
:e! return to unmodified file
**********************************************

********************************
/* HOW TO NAVIGATE IN FILE */
*********************************
gg to go to start of the program
G to go to the end of the program
`` to go to the previous position
~ to reverse the case of the letter under Cursor
0 to go to the start OF the line
$ to go to the end OF the line
Ctrl-u TO up the half the page
Ctrl-d TO down the half the page
Ctrl-f TO forward the page
Ctrl-b TO backward the page
H moves cursor to HOME of the page
M moves cursor to MIDDLE of the page
L moves cursor to BOTTOM of the page
:tabe or :tabnew to open new tab
:tabp - move to previous tab
:tabn - move to next tab
gt to move to next tab
gT to move to previous tab
3gt to move to 3rd tab
********************************

********************************
/*HOW TO SEARCH WORD IN FILE */
********************************
/word to find the word
/\<word\> to find exact word, not worda or worddd
* to find the exact word under the cursor forward
# to find the exact word under the cursor backward
gd to find the exact word under the cursor forward
gD to find the exact word under the cursor backward
g* or g# to find cursor word without \< and \>
J to join current line with line below with space
gJ to join current line with line below without space
?word to find the word
* to find the next
n to find next in (forward)
N to find next in (downward)
:g/ gives the lines with search word
:%s//new/g will replace the last search word with new globally
% in escapemode to find the other pair of the parenthesis
:linenumber to go to that line number
nG to go that line n
ngg to go that line n
:lvim /pattern/gj *.txt will search all txt files with pattern
:lvim /raj\_.\{-}shekar/gj **/* to search for raj and after shekar
:lvim /raj\_.*shekar/gj **/* to search for raj and after shekar
:lvim /\(raj\_.*shekar\)\|\(shekar\_.*raj\)/gj **/* to search for raj and shekar in any order 
:lw used after lvim to know the search results
:lgrep -i "raj" *.txt  search all txt files with raj
[I to search all the lines containing word under cursor
:lv /pattern/gj **/*.txt to search recursively 
/^pattern search all lines starting with pattern
/pattern$ search all lines ending with pattern
/^pattern.*raj.*shekar - to search line containing pattern as first word AND raj as next AND shekar in same line.
/pattern\_.*raj - to search for pattern AND raj with multiple lines between
/.*pattern\&.*raj - search for pattern and raj in any sequence in same line
/pattern\|raj - search for pattern or raj.
/\<\d\d\d\d\> - to search for 4 digit numbers
/.*word.* - to highlight all lines containing search word
:%s/word//gn - to count the number of occurences of word in file
:%s///gn - to count number of of occureneces of previous word
:promptf to find word in windows style
:promptr to replace word in windows style
*******************************

*********************************
/* HOW TO SAVE AND QUIT FILE */
*********************************
:q to quit
:q! to quit without save
:wq to save and quit
:w to write and be there in program
:x to write and exit
:x! not to write and exit
ZZ to save and exit in command mode (capital Z)
:w filename to write the content on file
:w! filename2 to overwrite the existing filename to filename2
**********************************

*********************************
/* HOW TO CUT,COPY,PASTE */
********************************
dd to delete
dw to delete the word under cursor
D to cut the line upto end of the line
yy or Y to copy the line (yank the line)
yw to yank current word under cursor
:1,10d -- to cut the lines in between 1 to 10 inclusive
:1,10y -- to yank/copy the text between 1 to 10 inclusive
:'a,'bd -- to cut the lines in between a to b marked tags inclusive
:'a,'by -- to yank/copy the lines in between a to b marked tags inclusive
:1,10 co 15 to copy all the lines from 1 to 10 (inclusive) after 15th line
:1,10 m 15 to move all the lines from 1 to 10 (inclusive) after 15th line
p to paste
shift+p to paste 
ddp to reverse 2 lines 
:t n -- copy current line after line n
:xt n -- copy line x after line n
:m n -- move the current line after line n
:xm n -- move the line x to line after line n
:r filename to paste that file at cursor position
cc to change the entire line 
cw to change the word under cursor
dG to delete from cursor line to end of file
dgg to delete from start of file to cursor line
ggdG to delete all contents in file
ggyG to yank all contents in file
d1G to delete from start of file to cursor line
nGdd to delete the line n
"bdd to yank or cut the line in one register b (it can be any letter from a to z)
"bp to paste the content in register b 
***********************************

**********************************
/* HOW TO INSERT TEXT IN FILE */
*********************************
a to insert mode
i to insert mode
A to insert at end of line
I to insert at first of line
o to insert in newline (next to cursor line)
O to insert in above line of cursor
:set ro readonly mode
:set ro! to remove readonly mode
********************************

**********************************
/* HOW TO UNDO */
*********************************
u to Undo
Ctrl+r to redo 
*********************************

***************************************
/* HOW TO MOVE OUT OF VI TO SHELL TO RUN UNIX COMMAND
**************************************
:shell goto shell
:sh goto shell
exit or Ctrl+d to return back to vi editor

:! command to execute that command while in VI
RETURN to come back to vi editor
*****************************************

******************************************
/* HOW TO REPLACE TEXT */
*****************************************
:%s/old/new/g  to replace old word/letter with new word without confirmation
:%s/old/new/gc to replace old with new with confirmation
:%s/\<raj\>/she/g to replace only "raj" with "she",,not rajashekar or raja 

:1,$ s/old/new/g to replace all old with new in whole file
explanation
1 -- start of file
$ -- end of file
s -- substitute
old -- old text
new -- new text
g -- globally

:n,m s/old/new/g to replace all old to new from line n to m
:%s/\s\+$//g to remove all white spaces at end of each line
:%s/^\s\+//g to remove all white spaces at start of each line
:%s/\r//g to remove all ^M in file
:bufdo %s/pattern/substitue/ge | update replace all pattern in files to substitute
:%s/suck\|buck/loopy/gc replace suck or buck with loopy
:%s/fred/<C-R>a/g to replace fred with contents of register "a"
************************************************

*************************
/ * TO CREATE .I FILE */ 
cc -save-temps filename
************************

*******************************************
/* SET OPTIONS */
******************************************
set options
:set ai for auto indent
:set si for smart indent
:set noai for no auto indent
:set nu for number display on left side of line
:set nonu for remove the displaying numbers
:set vb t_vb= to remove the beep sound 
:set hls for highliting search word
:set nohls for not highlighting the search word
:syntax on -- to on syntax of programming
:set incsearch to set increment search
:set ic to ignore case while search
:set backspace=2 to use backspace in insert mode
:set wrap to wrap up the lines 
:set nowrap to not to wrap up the lines
:set noerrorbells to remove the beep sound
:set errorbells to set the beep sound 
:set novisualbell to remove flash
:set visualbell to set flash
*********************************************

/*ESC*/
ctrl+[ is also called as escape mode
ctrl+n to complete the word 

*********************************************
/* HOW TO SPLIT SCREEN */
*********************************************
Split screen
:split anotherfile // horizontally spliting 
:sp filename // horizontally spliting
:vsplit anotherfile // vertically spliting
:vsp filename // vertically spliting

or at shell prompt
vi -o file1.txt file2.txt // horizontally
vi -O file1.txt file2.txt // vertically

at escape mode
ctrl-w-v to open a new window vertically
ctrl-w-n to open a new window horizontally

ctrl-w + increase the size of window
ctrl-w - decrease the size of window
ctrl-w = to equalize the size of each split window
ctrl-w | to maximize in vertical split
ctrl-w _ to maximize in Horizontal split
ctrl-w h to move to the window left
ctrl-w j to move to the window below
ctrl-w k to move to the window above
ctrl-w l to move to the window right
ctrl-w H to move current window to far right
ctrl-w J to move current window to far Down
ctrl-w K to move current window to far Up
ctrl-w L to move current window to far left
ctrl-w T to move current window to new tab
ctrl-w n to open new file in H split
ctrl-w q to close
ctrl-w o to close all splits except current window (:only also do the same)
***********************************************

*********************************************
TO KNOW THE STATUS OF THE PROGRAM
********************************************
ctrl-G 
:f 
***************************************

*******************************************
TO MARK SOME POSITION IN PROGRAM AND COMING TO BACK TO THAT PROGRAM
(MARKERS)
*********************************************
mp where the position is marked as p (it can be anything from a-z) 
to return back to that position p we type 'p

"ay'b to yank from current cursor line to the mark position b into register a
"ad'b to cut from current cursor line to the mark position b into register a
"ap to paste the content in that buffer
******************************************

******************************************
SETTING KEYS FOR SPECIFIC OPTIONS
*****************************************
nnormap <key> :set option<cr>
****************************************

*****************************************
TO CLEAN UP CONTROL M'S IN THE FILE 
*****************************************
:%s/<control v><control m>/ /g
*****************************************

***************************************
LOWER CASE TO UPPER CASE
*************************************
~    to switch case of charater under the cursor
g~~ to switch case of whole line
gUU - To make the whole line upper case
guu - To make the whole line lower case
U   - In visual mode make selected to upper case
u   - In visual mode make selected to lower case
******************************************

***************************************
INDENTING A FILE
**************************************
== makes the current line indent
In visual mode select the whole part and then == to indent the selected part
good settings
:set sw=4
:set smarttab
:set et 
******************************************

*******************************
GLOBAL COMMAND
*******************************
:g/pattern - list all the lines having pattern
Ctrl+x Ctrl+l gives list of lines to insert
:g/pattern/d - delete all the lines having pattern
:g!/pattern/d - delete all the lines that are not having pattern
:v/pattern/d - delete all the lines that are not having pattern
:g/pattern/ copy $ - copy all the lines that are having pattern to end of file
:g/pattern/ co $ or :g/pattern/t $ also do the same as above
:g/^$/d - delete all empty lines
:g/^\s*$/d - delete all empty lines precisely
:v/\S/d - delete all empty lines
:v/./d - delete all empty lines
:g/.*/m0 - This will reverse the order of the lines in the current file.
:g/^/m0 - to reverse the file
:g/^$/,/./-j to replace mulitple empty lines with single line
:v/./,/./-j to replace mulitple empty lines with single line
:g/pattern/ .w! >> newfile.txt - writes all line having word pattern 
:.w! >> newfile.txt - writes current line into newfile
:g/pattern/ copy $ - to paste all the lines containing pattern to end of file
:g/pattern/ t $ - same as above
:g/pattern/ t 5 - to paste all the lines containing pattern after line 5
:g/start/,/end copy $ - copy from start to end to end of file
:g/start/,/end/d - to delete start to end block
:g/start/,/end/j - to join start to end.
:g/start/y A - to copy all the lines to register A
:g/^pattern/s/$/end - Add end for lines starting with pattern
:g/^/pu _ to double space the whole file
:redir @r - redirect to register r
:g/pattern - search globally
:redir end 
"rp paste the contents in register r
*************************************************************

*****************************************
MORE USEFUL COMMANDS
*****************************************
:Ex opens explorer in seperate buffer 
:Sex open explorer in split
:Vex open explorer in vertical split
:Tex open explorer in new tab
:browse e open explorer in windows style to open
:bro op open explorer in windows style to open 
:bro sav open explorer in windows style to save
Ctrl+n (in insert mode) gives list of matching words to insert
Ctrl+p (in insert mode) gives list of matching words to insert
Ctrl+x Ctrl+l gives list of lines to insert
ggVGg? to rot13 the whole file
ggg?G to rot13 the whole file
********************************************

*******************************************
FUNCTIONS example
*******************************************
use TABS to complete words 
function! Tab_Or_Complete()
  if col('.')>1 && strpart( getline('.'), col('.')-2, 3 ) =~ '^\w'
    return "\<C-N>"
  else
    return "\<Tab>"
  endif
endfunction
:inoremap <Tab> <C-R>=Tab_Or_Complete()<CR>
:set dictionary="/usr/dict/words"
*******************************************

*******************************************
Visual Mode
*****************************************
v - visual mode
V - visual mode to select whole line
gv - select last visual select
o - navigate in visual select
V% - to select match
V}J - join the selection
zf - in visual mode, fold selection
za - to open and close fold
zo - to open the fold
zc - to close the fold
zR - to open all folds
zM - to close all folds

Ctrl Q - to select columns (Visual mode)
Ctrl V - to select columns (Visual mode)
********************************************

*******************************************
DIFFERENCE 
******************************************
:vsp filename2 - (in filename1)
:windo diffthis - (find diff between filename1 and filename2)
do - to obtain the difference for the current line from diff file
dp - to put the difference for the current line to diff file
********************************************

********************************
MAPPING
********************************
map <F11> : set hls! <Bar> set hls? <CR>
<CR> : Carriage Return
<ESC> : Escape
<LEADER> : \
<BAR> : |
<BACKSPACE> : backspace
<SILENT> : no hanging shell window
********************************

********************************
ENCRYPTION 
********************************
:X to set password for file (powerful)
:set key= to remove password for file
***********************************

*********************************
REGISTERS
********************************
:reg to show the registers
"ay yank current line to register a [26 registers]
"ap paste buffer a
[A-Z] registers are to append
[a-z] registers are to override
qaq to clear the register a - record nothing
Ctrl-r register - to paste the contents of the register in insert mode
*************************************

*************************************
BUFFERS
*************************************
:ls to list all the buffers 
:bp to move to previous buffer
:bn to move to next buffer
:bd to remove from buffer
:args **/* to keep all files in the current directory in vim (SUPER)
:rew to first file in args
:te Buffers to show the menu buffer
:badd add file to buffer
:b# to alternate buffer
Ctrl-^ is to switch alternate buffers
:tab ball to open all buffers in tabs
:ball to open all buffers Horizontally 
:vert ball to open all buffers vertically
:xa save all buffers and exit
:wa write all buffers 
********************************

********************************
RECORDING
********************************
qq is to start recording commands - use (a-z) to record
@q to execute command in recording
n@q to execute command n times
qqq to remove recorded command
********************************

********************************
ABBREVIATION
*******************************
:abbr USA United States of America - to set the abbreviation
type USA (in insert mode) space will give you USA.
:unabbr USA to unset the abbreaviation
********************************

********************************
SPELL CHECKING
********************************
:set spell - to on spell checking
:set nospell - to off spell checking
z= to get the spell corrections
[s to previous misspell
]s to next misspell
*********************************

*******************************
INDENTING XML
*******************************
nJ - join n lines with one space in between
ggVGJ - to join all the lines with one space in between 
ggVg :j - to join all the lines with one space in between  
ggVg :j! - to join as it is
:%s/></>\r</g
:set filetype=xml
gg=G or ggVG= 
*******************************

*******************************
INCREMENTING AND DECREMENTING 
*******************************
:nput =range(1,10) - puts 1 to 10 numbers in incrementing order at line n
:nunmap <C-A> will make Ctrl A (in windows to non map from SELECT ALL)

qq start recording in q
yy yank the current line
p paste to next line
Ctrl A to increment (Ctrl x to decrement)
q to stop the recording
12@q to repeat the recording

:for i in range(1,10) | put = 'insert into'.i | endfor
to add line numbers
:g/^/exe ":s/^".line(".")." . "
exe will execute strings in ""
. is used to concat'
line(".") gives the number of current line
*******************************

***********************************
In insert mode
***********************************
Ctrl-n or Ctrl p - for matching words under cursor
Ctrl-w to delete the word under the cursor
Ctrl-r register - to paste the register contents
Ctrl-w l or Ctrl-w Ctrl-l to match the lines under the cursor 
***********************************

***********************************
SORTING
***********************************
:sort to sort alphabetically
:sort! to reverse the order
:sort n to sort numbers
:sort /pattern/ - sort based on pattern very powerful
***********************************

***********************************
Tracing
***********************************
`` to go back to last position
'. to go back to the last modified line
`. to go back to the last exact position
C-o to move back to previous positions (OLD)
C-i to move front positions if any (new)
:jumps to know the movements
:history to know the previous commands
***********************************

***********************************
Using back reference while substitue
***********************************
\(^.*$\) - select whole line 
:%s/\(^.*$\)/blah \1 blah/g - *POWERFUL* on every line add blah at first and last
:%s/\(raj\)\(shekar\)/\2\1/g - Swapping 2 numbers with help of back reference
:%s/raj/&shekar/g - to replace raj with rajshekar - '&' to substitute search term.
***********************************
***********************************
Editing at column
***********************************
Ctrl+q - column mode
/\%25c - select 25th column
/\%>25c - select all after 25th column
***********************************

***********************************
Finding duplicate lines
***********************************
/\(^.*$\)\n\1 - to find duplicatel lines
:sort u  - removing duplicate lines
***********************************