Friday, March 3, 2017

VIM - Text Editor Tips

I've compiled few tips over time on vi which I found very useful and little harder to get on the net.

Tip - Configure your vim
On Linux you configure ~/.vimrc file or set VIMINT variable
On Windows its easier to edit this file by opening a vim session and typing
  :edit $MYVIMRC #Start vim with ADMIN privs to edit this file
To figure out what ~ aka home dir is on windows you can do
:echo expand('~')
Few other things you can print out while within vi are
:echo $HOME
:echo $VIM
:echo $VIMRUNTIME
:version

Tip - Getting help
vimtutor #An executable installed with gvim to help you get started on vi
:help
:help subject
Jump to a subject:  Position the cursor on a tag (e.g. |bars|) and hit CTRL-].
Jump back:  Type CTRL-T or CTRL-O.  Repeat to go further back.

Tip - Relocate ~ tilde files
One thing you don't want vim to do is to clutter your directory with ~ tilde
files, especially if you've source controlled the folder. To avoid that place
few settings in this file thus:
(on windows)
set backupdir=c:\\temp\\vim//
set undodir=c:\\temp\\vim/
set dir=c:\\temp\\vim//

Tip - Working with tabs
set tabpagemax=100 #maximum tabs you can open
"C:\Program Files (x86)\Vim\vim80\gvim.exe" -p * #Open all files under the
directory in tabs!

I find the most convenient method of editing multiple files is by using tabs. You can open multiple files in separate tabs via the command line like so:

vim -p file1.txt file2.txt

Or if you already have vim open, you can open a new file in a new tab like so:

:tabe file2.txt

Once you have the tabs open, use gt to view the next tab and gT to view the previous tab.

You can also jump to the first tab with 1gt, the second tab with 2gt, etc.

You can close tabs using :tabc

Finally you can move the current tab to the nth location with :ntabm where n is any number greater than or equal to 0.


Tip - Working with indentations, useful for coding python
First set your variables
:set wrap linebreak nolist ai si nu ic sw=4 ts=4 expandtab softtabstop=4
#wrap Sets line wrap if you're window is small ,then linebreak breaks at word
boundary not in middle of word.
#list display tab with CTRL-I and endofline as $, nolist disables that
behavior.
#ai autoindent , new line starts with similar indentation as earlier line
#si smartindent, useful for C type langs, start indents after line ending with {
#nu enable line number; nuw 8 sets the width to 8 chars for line nu
#ic ignore case
#sw shiftwidth ,default of 8 chars , when indenting with '>', use 4 spaces width
#ts or tabstop=4 show existing tab with 4 spaces width
#On pressing tab, insert 4 spaces ,as a result of expandtab and softtabstop

You can replace all the tabs with spaces in the entire file with
:%retab

4>> , 4<< #For indentations, move next 4 lines  and indent them
= #Autoindent, looks at indentation of line above and matches lower lines with it.
  # =G indent till end of file
  # =5 five line after this
  # =ap a paragraph?



Tip - Working with Splits
"C:\Program Files (x86)\Vim\vim80\gvim.exe" -O3 *  #Open all files in 3 vertical splits!

:split filename  - split window and load another file; :sp
:vsplit file     - vertical split; :vs
:e filename      - edit another file
:e# toggle files
'A-Z marker between files.

ctrl+w (hjkl) move between windows
 ctrl-w up arrow  - move cursor up a window
 ctrl-w ctrl-w    - move cursor to another window (cycle)
 ctrl-w_          - maximize current window
 ctrl-w=          - make all equal size
 10 ctrl-w+       - increase window size by 10 lines

 :sview file      - same as split, but readonly
 :hide            - close current window
 :only            - keep only this window open
 :ls              - show current buffers
 :b 2             - open buffer #2 in this window

You can use the option -o to open the files in horizontal splits or -O to open vertical splits. The following commands open a window for each file specified:

vim -o *.cpp
vim -O foo bar baz

You can tell Vim the maximum number of windows to open by putting an integer after o or O options, the following example will open at most two windows no matter how many file matches, you will see the first two file specified on the command line, the rest will remain hidden:

vim -o2 *.cpp

See :help -o for all the details.

:args app/views/*.erb | all    #for horizontal splits or

:args app/views/*.erb | vertical all   #for vertical.


Tip - Command Line Mode
First /  to Go in clm search OR : to go in clm commands
 ctrl+p show previous historical search
 ctrl+n next search
ctrl+f

q/ history of search patterns
q: history of commands

Tip - Multiple Searches in vim 
I'm not sure if there's any editor that can do this, we can search for multiple separate words in a single search, here's couple of mechanisms for doing that:
/w1\|w2\|w3
\vw1|w2|w3

If lets say you want many words highlighted then place this in your .vimrc file - really good way to always highlight few keywords you want. (Ref 9,10)
:match Float /weblogic.home\|-Xmx\|-Xms\|weblogic.Name\|config.dir/


Tip - Handle Multiple Files
You can open multilple files simply by vi a.txt b.txt etc. or lets say you did
a grep and found many files as a result you want to go over ,then you can use
something like this to open all at once:
vi $(!!) ##Output of last command
vi $(grep -il "webcat" *.log)
$ find -name testname.c -exec  vi {} +

:n
:N
:args  #shows all files opened with current in brackets.
:rew
:help buffer
Ctrl+g #what file
 :ls              - show current buffers
 :b 2             - open buffer #2 in this window


Tip - Copy and paste to a different file within vim opened files
ma #mark the line
go to other line
y`a #yank from the marker to current line
:ex test.txt #open a new file
p #paste it
:rew #go back to old file

Alternatively:
ma
mb
`a,`byz #y->yank z->buffer
"zp #buffer paste



Tip - Pasting text without messing up formatting

:set paste  #auto indent destroys paste formatting
:set nopaste
set pastetoggle=<F2> # to easily toggle setting
"*p or "+p for pasting from clipboard.

Tip - Vimdiff
Sorry - but I typically use kdiff for better visualization !

Tip -  converting from \r\n to \n only
http://vim.wikia.com/wiki/File_format
:argdo set ff=unix|w

Tip - Highlight cursor
:autocmd CursorMoved * silent! exe printf('match IncSearch /\<%s\>/', expand('<cword>'))
:set hlsearch


Tip - Stop messing with the color and syntax
:syntax off
:set syntax=
syntax off
set nohlsearch
set t_Co=0  #line number color off
set t_Co=0  syntax=strace

Sometimes colors are useful, I typically use zellner which works on most terminals
:colo zellner

Tip - Miscellaneous

gUaw # Uppercase a word
* #search for current word which cursor is on

/echo.*file #Search with pattern
 :%s/^\(Martin\)/Mr \1 Wicks/g  #Group search and replace
4!!awk '{print "New text",$0}' #Appends New text to front of line for 4 lines. you can do this with substitute ^ too, where awk is not needed.
:1,$!awk '{print $1}' | sort | tr [:lower:] [:upper:]  #Just get first column from file sorted and in upper case; replace $1 with $0 to sort have all fields;
:.,$s/^/\t/  #substitue front of line with tabs.
:1,$s/^M//
:s/ \+/\r/g #Break from spaces to newlines

Show special characters
 https://yakking.branchable.com/posts/whitespace-safety/
http://www.shsu.edu/~csc_tjm/fall2000/cs431/shellprog.html
Just to clarify, :set list shows newline ($), :e ++ff=unix shows CR (^M); if you want to see both, :set list then :e ++ff=unix
 or :set binary or vi -b file
 vi -b myfile.txt


Delete

To delete forward up to character 'X' type dtX
To delete forward through character 'X' type dfX
To delete backward up to character 'X' type dTX
To delete backward through character 'X' type dFX



daw #delete a word ; even if you're at middle of word
daw
daW
diw #insert mode
g!/TEXT/d - deletes lines which don't have TEXT in it.. note the !
:g/^$/d - Removes blank lines; Trim lines ; remove duplicate lines  :sort  u ;



Tip - Pad + with spaces - From Ref. 8
f+   #find +
s     #delete char at cursor i.e + and go in insert mode
_+_  #pad + with spaces
;    #repeat find with f
.   #repeat last sequence i.e pad +

 Folding
 http://vimdoc.sourceforge.net/htmldoc/fold.html
 https://www.linux.com/learn/vim-tips-folding-fun
 http://stackoverflow.com/questions/8316139/how-to-set-the-default-to-unfolded-when-you-open-a-file

 zR #Open all folds
 zM #Close all folds
 zi #Toggle.
zf#j creates a fold from the cursor down # lines.
zf/string creates a fold from the cursor to string.
zj moves the cursor to the next fold.
zk moves the cursor to the previous fold.
zo opens a fold at the cursor.
zO opens all folds at the cursor.
zm increases the foldlevel by one.
zr decreases the foldlevel by one.
zd deletes the fold at the cursor.
zE deletes all folds.
[z move to start of open fold.
]z move to end of open fold.

Tip - Good Links for VIM
http://www.vim.org/  #Main site - download from here

http://vimhelp.appspot.com/vim_faq.txt.html
https://www.cs.swarthmore.edu/help/vim/markers.html 
 
 
Tip - Sessions in vim
 :mksession ~/sess.vim
:source ~/sess.vim
gvim -S ~/sess.vim
--save with :xa or :wqa
-might want to set this in .vimrc
set ssop-=options    " do not store global and local values in a session
set ssop-=folds      " do not store folds
 
Tip - Join multiplie lines
J- Joins next lin
[range]j[lines]
%j - joins whole buffer.
%j! - don't replace newlines with spaces..

Tip - Keep portion of text from lines 
%s/\(\[.*, \).*/\1/g
-In \( give pattern to match , then end with \)..then for replace,put that remembered pattern with \1
 
Tip - Working with Macros 
qd     start recording to register d
...     your complex series of commands
q     stop recording
@d     execute your macro
@@     execute your macro again 
In the link above the first command should be  :s/\s\+/': ' (then press Enter)
 
References Used
  1. http://vimregex.com/
  2. http://www.ibm.com/developerworks/aix/library/au-vitips.html
  3. http://stackoverflow.com/questions/1551231/highlight-variable-under-cursor-in-vim-like-in-netbeans
  4. http://alvinalexander.com/linux/vi-vim-editor-color-scheme-colorscheme
  5. http://vim.wikia.com/wiki/How_to_turn_off_all_colors
  6. http://vimcasts.org/episodes/refining-search-patterns-with-the-command-line-window/
  7. http://vim.wikia.com/wiki/A_better_Vimdiff_Git_mergetool#Vimdiff_for_three-way_merges
  8. Practical Vim, 2nd Edition By Drew Neil
  9. http://stackoverflow.com/questions/704434/is-there-any-way-to-highlight-multiple-searches-in-gvim
  10. http://stackoverflow.com/questions/4162664/vim-highlight-a-list-of-words#comment4494852_4162735 
  11.  https://missing.csail.mit.edu/2020/editors/
  12. https://stackoverflow.com/questions/1642611/how-to-save-and-restore-multiple-different-sessions-in-vim


Creating a new Oracle Linux Virtualbox Virtual Machine for personal use

I'll take you though creating an Oracle Linux 6.5 VM on a Windows PC. This was the latest Linux available at the time.
Creating a VM gives you all the benefits from my earlier post. Also, you can build your own image from latest linux builds and configure it the way you want to.

Install latest VirtualBox and its guest additions on your local PC from its website.
Download the OS Image Oracle Linux 6.5 from edelivery site https://edelivery.oracle.com/linux/ . It will be in an ISO File.

In Oracle VM VirtualBox Manager, File -> Preferences -> General -> Choose Correct Default Machine Folder. Example, E:\OVM , especially if you’re creating VMs on a different disk.

In Oracle VM VirtualBox Manager ,choose New

Create a NEW machine, enter a name you want to give


Allocate enough memory to VM as needed. Typically 2GB should be good enough for many tasks in Linux, but if you want to develop in NetBeans for example you should give around 4GB.

 Choose do not add a virtual hard drive yet. We’ll add those later. Hit Create. On the listed newly created VM, select it and choose Settings, to modify its settings.

In Settings go to Storage options first and near Controller:SATA hit the icon to add disk.

Choose VDI file type. This format has benefit of compacting files and resizing.

 On next screen choose “Dynamically Allocated” physical hard drive.
Next,  choose file location and size. Give a good name (name_DATA) and location and select around 100 GB or as seen necessary for your work.
Add another disk (name_SWAP) for swap partition on linux. Allocate 8 gb file. I chose dynamically allocated for swap, but if performance is of concern you can choose fixed type for better performance. You’d have to wait for the big file to be created though.
Next In  Settings -> storage ->  Controller:IDE , CD/DVD Drive. Load up the ISO by clicking the cd icon next to drop down.

Hit OK. Start the VM by double clicking or hitting Start.
Virtual Box starts up the Linux installation process.

Hit Ok on this message, once you get Linux fully installed it won’t affect you.
  Follow the upcoming menus
  Choose ntp time server
  Choose Create Custom Layout  option, instead of use all space option, so that you can layout swap in other 6gb partition.


Create standard partition on the large 30gb hard drive, give its mount point as /


 On other drive choose swap as the file system type and click fill to maximum allowable size.

Choose Software Development Workstation installation option; Note that basic server option doesn't give gui and is useful for server class systems; desktop option also had issues in installing VB guest additions.


Create group dba, user oracle by using advance user creation option.

  Reboot your VM
You should Install guest additions as it gets rid of the mouse locking , help resize your window automatically .
Once the Linux VM is up, do an  yum update ; repeat if needed. You’re now all set to install your favorite software on it.

Please let me know if you found this useful, It took me some time to figure this out even with help of a good reference as mentioned below..

Author: Dhiraj Soni
Date: Feb 4, 2014

References:
VirtualBox - Creating a Base OEL Image, by Jim Fisher   Nov 4, 2010

Thursday, March 2, 2017

VirtualBox - a virtualization technology


One of the technologies I find hard to live without is the virtualization technology. On daily basis I utilize Linux OS virtual machine(VM) running on my Windows Laptop. If you're not using this technology yet, perhaps this article can help you look into utilizing this.

Virtualization is seen as one of the enabling technologies of this generation. It enables much better utilization of the huge processing power of machines now available. Still, this was largely seen as domain of enterprise computing not at a laptop level. With advent of laptops of 4GB+ ram, it became more feasible to utilize virtualization at desktop/laptop levels. One such enabling technology is using open source VirtualBox (VB) virtualization (https://www.virtualbox.org/). This is comparable to VMWare if you're familiar to that.

With VB a key benefit is you get a self contained virtualized OS VM platform which you can move around with ease on any other PC you have. The virtual OS could be of many flavors like Windows, Solaris, Linux and Mac OS X. You can install any software on that VM and work with it like you do on a native OS. 

You can possibly work with multiple VMs on a single laptop, and you can also relocate the full working environment to other laptops with ease. You don't have to go though painful process of reinstalling your software on OS on other laptop where you're moving to.

Another benefit is to have a snapshot of the VM available, this is a point in time of the current state of your OS and you can revert back to this state later. This allows you to install software and if it doesn't work then you can revert back your VM state of the last snapshot. This saves a lot of time and effort if something goes wrong.

These days many vendors like Oracle give out pre-built VMs with their software installed on it. So you save lot of time by just reusing their VMs as your starting point for your development work. Few pre-built developer VMs are available here at oracle site.

VirtualBox’s many other benefits can be found in its very useful documentation. You can read through this manual to get a good understanding on how to go about setting up and using this technology.

YUM - Linux tool for managing updates.

9/4/2020:  DNF - Dandified yum is the next generation version of yum. See these links:
Commands work just like yum.
 
 
 
Yum is a package installer/remover for rpm systems. It can also update your system and is used mainly by the Linux systems such as Oracle Linux to simplify these tasks.
Some simple tasks using yum is to list,install, search, remove the packages on your system.

To list packages :
  • yum list installed  #list installed packages
  • yum list #Just list all
  • yum list |grep ^java  #find in the list
  • yum list 'tk*' #find in list

To remove a package
  • yum remove pkg #this will remove dependencies as well 
    yum erase pkg #similar to above
     
To install a package(s)
  • yum install pkg eg. yum install ksh
  • yum install libpng-devel ; gd-devel; openmotif-devel ; zlib-devel; expat-devel 
  • yum install gtk2-devel pango-devel glib2-devel pygtk2-devel python-devel
    libXt-devel  zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel rea
    dline-devel tk-devel gdbm-devel libpcap-devel xz-devel libX11-devel libpng-devel
      gd-devel openmotif-devel  zlib-devel expat-devel libglade2-devel numpy pycairo
    -devel pygobject2-devel expect perl-CPAN

To update package

  • yum update pkg
  • yum update-minimal --skip-broken 
If you just do yum update without pkg, then it will update entire system.

Few other very useful commands:
  • yum info package # Gives info on the package similar to rpm –qi
  • yum install yum-utils  #this package gives yumdownloader utility to download but not install packages.
  • repoquery --list package_name #Lists all file in package
  • yum provides "*bin/named"  #Find what package provides the given value
  • yum search gtk #Searches any package thus named 


Yum's configuration file contains other useful info. Eg. To setup a proxy for yum ,if you go over vpn do
vi /etc/yum.conf
proxy=http://host:80 
 
Yum can also work with group of packages. yum grouplist will display list of installed and available package groups. yum groupinfo GRP will display info on that group
.
Any changes yum does to your system gets recorded in a file ,most likely at location /var/log/yum.log . This is very useful to track changes .

" When a package is downloaded, installed and is removed there is a chance that the package may still be saved/stored in the yum’s cache. So to clean all the cached packages from the enabled repository cache directory, login as root and execute the following:
yum clean packages
To purge the old package information completely, execute the following command:
yum clean headers
To clean any cached xml metadata from any enabled repository, execute the following
yum clean metadata
If you wish to clean all the cached files from any enabled repository at once, execute the
yum clean all
" From Ref 6



PackageKit issue- This is where yum list complains that it is already running and locked by other process.

You should wait for it to clear up, if you know for a fact process is not running and is an issue, you could just kill the process as mentioned its pid.
Package kit can also be disabled in gnome via System > Preferences > Startup Applications

Dependencies issue
Rare cases few packages don't get installed due to latest version already available, if you need older version of it you need to remove that one.. Removing packages thus is not recommended, but can be done thus:
  •   rpm -e --nodeps graphite2-1.3.10-1.el7_3.i68
  •   rpm -e --nodeps graphite2-1.3.10-1.el7_3.x86_64


Instructions for manually connecting to Oracle Linux Yum Server can be found here:
http://public-yum.oracle.com/getting-started.html

/etc/yum.repos.d -> has repositories mentioned 

To get further help you can always do  yum -h , or man yum.




Finally, few good links on yum for further information:
  1. http://yum.baseurl.org/  #Yum Site
  2. http://docs.oracle.com/cd/E37670_01/ #OL6 documentation
  3. http://www.centos.org/docs/5/html/yum/index.html #YUM in Install guide
  4. http://www.thegeekstuff.com/2011/08/yum-command-examples/
  5. http://linuxgazette.net/issue68/nazario.html ##RPM
  6. https://kb.hivelocity.net/how-to-clear-the-yum-cache/
  7. http://public-yum.oracle.com/
  8. https://blogs.oracle.com/wim/entry/setting_up_oracle_linux_6
  9. Book: Practical guide to Linux Commands, Editors and Shell Programming by Mark G. Sobell. 

 

RPM

https://www.tecmint.com/20-practical-examples-of-rpm-commands-in-linux/

https://www.cyberciti.biz/howto/question/linux/linux-rpm-cheat-sheet.php