Thursday, September 10, 2020

tmux multiplexer

 Used for enhancing your work area in linux command line.

Useful Links:


yum install tmux

Tmux has sessions, windows and panes hierarchy.

tmux -V #Version
    new #new session
    new -s [name of session]
    ls #list sessions
    attach-session -t 3
    a # #attach last session with #
    a -t [name of session] #reattaches to the session.
    rename-session -t 0 database
    kill-server

    
    
Ctrl + b

  •     ? - help
Sessions
  •     d - detach ; D - which session to detach; 
  •     s sessions; ) , (  , switch sessions;  
  •     new #new session
  •     new -s [name of session]
  •     ls #list sessions
  •     attach-session -t 3
  •     a # #attach last session with #
  •     a -t [name of session] #reattaches to the session.
  •     rename-session -t 0 database
  •     kill-server

Windows
    •     c - create window;  
    •     switch windows; by n,p  <number>;     n(next), p(previous), number
    •     , - rename window
Panes
    •     " - split horizontally; display split into 2 different panes.
    •     % - vertical split
    •     [arrow key] to move between panes
    •     [HOLD arrow key] to resize  panes
    •     x kill current pane
    •     exit or ctrl_d exit a pane
    •    z - full screen pane and back;
    •     :resize-pane -D 2 #-U for up, -D for down -L for left and -R for right. The last part is the number of lines to move it over by.

  •     x - kill
  •     :list-keys

    
    ~/.tmux.conf
    
Copy paste; hold Shift key then select from mouse

If you press shift while doing things with the mouse, that overrides the mouse protocol and lets you select/paste. It's documented in the xterm manual for instance, and most terminal emulators copy that behavior.


https://gist.github.com/spicycode/1229612

oracle@localhost $ cat .tmux.conf
# Uncomment the lines with the options you want to activate (by deleting the preceding "#")

# Allow mouse interaction
#setw -g mode-mouse on
set -g mode-mouse on

# Allow mouse to select which pane to use
set -g mouse-select-pane on

# Allow mouse dragging to resize panes
set -g mouse-resize-pane on

# Reload tmux config
bind r source-file ~/.tmux.conf

#set -g status-right '#[fg=yellow]#(cut -d " " -f 1-3 /proc/loadavg)#[default] #[fg=white]%H:%M#[default]'

On unix you can also use alternatives like screen or  gcm.

Tuesday, June 30, 2020

Migrate Mediawiki to new version on westhost platform

Westhost ( https://www.westhost.com/ ) is my long time website hosting platform, recently upgraded their platform to utilize cpanel. They moved the mysql databases and site to new platform, but the wiki's i've been running earlier stopped working. I haven't been actively managing those wiki's so the software was quite old version 1.14 of Mediawiki than current 1.34.2
The platform also changed in many ways. Newer php 7.2.x, mariadb, cpanel.
So the old wiki's refused to work with newer php, and the sites went down!
I was left with many questions , in order to get sites working.
Primarily, How to make Mediawiki work with new php, and get the db also updated.?
So, this is what i tried:
Download new software-
https://www.mediawiki.org/wiki/Download
Read through its upgrading doc at  https://www.mediawiki.org/wiki/Manual:Upgrading
It looked like many LocalSettings.php(LS) parameters also changed, and i had no idea what would work. So, first thing, instead of upgrading existing wiki's. I installed a new wiki altogether, utilizing the install guide at https://www.mediawiki.org/wiki/Manual:Installation_guide#Main-installation-guide

The westhost's cpanel has ui for Mysql DB's - so created new Test db there and associated a new user to it. Cpanel's Metrics->Errors was useful to veiw errors as they happened. For some reason , apache's direct access to its log is not there in new platform?

So, after installing a new wiki, it had generated its new LS , which was now useful to compare to the very old wiki's parameters.

In order to upgrade, encountered quite a few issues, i'm documenting steps below to help anyone doing similar excercise:

Do a  http://www.yourdomain.com/phpinfo.php -> To see if php runs and what version.
extract mediawiki s/w
 wget https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.2.tar.gz
rename old wiki to wiki_old.. rename mediawiki to wiki
chmod -R g-w wiki
 cd wiki
 chmod -R 644 *.php */*.php

change LocalSettings parameters carefully
 add .htaccess with just -> DirectoryIndex index.php
copy old images, skins/common files from old wiki to new. 
Go to wiki/maintenance and run "php update.php" command. This fixes the DB to work with new version of mediawiki. ##IF this does not work try the web update..http://www.yourdomain.com/mw-config  ..that shows errors happening.. For me few extensions were outdated ,so without any error message , i couldn't figure out what's wrong. So used the web update, which showed those errors.


Do enable display errors feature from cpanel ->  Multiphp ini editor section -> display_errors; prior

Once update is done, the wiki should work normally. Thanks to Mediawiki folks for documenting so well, and making the s/w update smooth. I didn't think it would be possible to upgrade from such a old version.

LS parameters updated:
wgServer
Database settings
$wgDBTableOptions   = "ENGINE=InnoDB, DEFAULT CHARSET=binary";
#$wgDBmysql5  commented this
$wgMemCachedServers = [];
wfLoadSkin( 'MonoBook' );
wfLoadSkin( 'Timeless' );
wfLoadSkin( 'Vector' );
Extensions to:
wfLoadExtension( 'Cite' );
Comment/remove the $wgUpgradeKey  once the update is done.

Tuesday, March 20, 2018

Adding a Custom search URL to your firefox browser

It is a very common that we have a website which we enter a search term regularly to get response back. Instead of going to website and use its search feature, we can directly route that search to website via FF search.

So, how to make that work?

First read up on the link here which details the process : https://developer.mozilla.org/en-US/docs/Talk%3ACreating_OpenSearch_plugins_for_Firefox

Then create two files one .htm and another .xml . You open the .htm file in browser and the search icon changes with a +, indication a search engine is available on website to be added to browser.

Now ,if the files are on you local laptop, the url localhost won't work, Also, FF blocks file:/// urls, so what to do?  Simplest way it to run a local webserver, you can do it variety of way, one of way is to just run  local webserver from the directory where these files are contained: Example using python:

python -m SimpleHTTPServer
(for v3) python -m http.server [<portNo>]


Example of .htm

<html>
    <head>
        <link rel="search"
      type="application/opensearchdescription+xml"
      title="YOURWEBSITE"
      href="http://localhost:8000/YOURWEBSITE.xml">
        YOURWEBSITE
        </link>
    </head>
</html

Example of .xml; replace with your own URL, and customize accordingly.

<?xml version="1.0" encoding="UTF-8" ?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
     <ShortName>YOURWEBSITE</ShortName>
     <Description>YOURWEBSITE)</Description>
     <Image width="16" height="16">data:image/x-icon;base64,R0lGODlhEAAQAOMAAAAAABgYtRgYrRgYlBBSMcaEMQhKKTFaKcZ7Mb17MRBKKYxzKQAAAAAAAAAAAAAAACH5BAEAAAAALAAAAAAQABAAAARGEMhAq7y36iBwphjXhaAnnKR3DUOqAqyrxt8Lt18w05fAYawBQeULBguGA9E4MCASB4XtYnAuDsopoJqASrVVBMKglVSrEQA7</Image>
     <InputEncoding>UTF-8</InputEncoding>
     <Url type="text/html" method="get" template="https://YOURWEBSITE.com/xyzurl/name?searchName={searchTerms}"/>
</OpenSearchDescription>



That's it to it you got your search engine available on FF. Enjoy!

Saturday, February 24, 2018

Getting Started with graphing utilities

In this post we look at few graphing utilities which would help us in rendering graphs (charts) simply. Few of the tools I researched are discussed here.
I'm still looking out for a good tool to have these charts zoom in/out and ability to focus in on small time periods. Let me know if you know ?

Python - Pygal

Pygal is a really good utility to generate graphs in svg or png with default rendering being good enough.

Built a tutorial youtube video on this which can be used to get going on this tool.

The code files for that same video can be found here on GitHub

 

 Python – MatPlotLib & Mpld3


 This is a also a powerful tool to render graphs. Mpld3 is used to get the rendering for browsers and clean up the default rendering by matplot library. By default didn't like rendering as much as pygal above.

Built a tutorial youtube video on this which can be used to get going on this tool.

The code files for that same video can be found here on GitHub

 

GnuPlot

This cool tool allows a powerful rendering of charts without much coding at all. Mostly configuration needs to happen and it can read data files on its own and render graphs for you. Very powerful in its own right and really good rendering.


Built a tutorial youtube video on this which can be used to get going on this tool.

The code files for that same video can be found here on GitHub


Friday, March 3, 2017

VIM - Text Editor Tips

I've compiled few tips over time on vi which I found very useful and little harder to get on the net.

Tip - Configure your vim
On Linux you configure ~/.vimrc file or set VIMINT variable
On Windows its easier to edit this file by opening a vim session and typing
  :edit $MYVIMRC #Start vim with ADMIN privs to edit this file
To figure out what ~ aka home dir is on windows you can do
:echo expand('~')
Few other things you can print out while within vi are
:echo $HOME
:echo $VIM
:echo $VIMRUNTIME
:version

Tip - Getting help
vimtutor #An executable installed with gvim to help you get started on vi
:help
:help subject
Jump to a subject:  Position the cursor on a tag (e.g. |bars|) and hit CTRL-].
Jump back:  Type CTRL-T or CTRL-O.  Repeat to go further back.

Tip - Relocate ~ tilde files
One thing you don't want vim to do is to clutter your directory with ~ tilde
files, especially if you've source controlled the folder. To avoid that place
few settings in this file thus:
(on windows)
set backupdir=c:\\temp\\vim//
set undodir=c:\\temp\\vim/
set dir=c:\\temp\\vim//

Tip - Working with tabs
set tabpagemax=100 #maximum tabs you can open
"C:\Program Files (x86)\Vim\vim80\gvim.exe" -p * #Open all files under the
directory in tabs!

I find the most convenient method of editing multiple files is by using tabs. You can open multiple files in separate tabs via the command line like so:

vim -p file1.txt file2.txt

Or if you already have vim open, you can open a new file in a new tab like so:

:tabe file2.txt

Once you have the tabs open, use gt to view the next tab and gT to view the previous tab.

You can also jump to the first tab with 1gt, the second tab with 2gt, etc.

You can close tabs using :tabc

Finally you can move the current tab to the nth location with :ntabm where n is any number greater than or equal to 0.


Tip - Working with indentations, useful for coding python
First set your variables
:set wrap linebreak nolist ai si nu ic sw=4 ts=4 expandtab softtabstop=4
#wrap Sets line wrap if you're window is small ,then linebreak breaks at word
boundary not in middle of word.
#list display tab with CTRL-I and endofline as $, nolist disables that
behavior.
#ai autoindent , new line starts with similar indentation as earlier line
#si smartindent, useful for C type langs, start indents after line ending with {
#nu enable line number; nuw 8 sets the width to 8 chars for line nu
#ic ignore case
#sw shiftwidth ,default of 8 chars , when indenting with '>', use 4 spaces width
#ts or tabstop=4 show existing tab with 4 spaces width
#On pressing tab, insert 4 spaces ,as a result of expandtab and softtabstop

You can replace all the tabs with spaces in the entire file with
:%retab

4>> , 4<< #For indentations, move next 4 lines  and indent them
= #Autoindent, looks at indentation of line above and matches lower lines with it.
  # =G indent till end of file
  # =5 five line after this
  # =ap a paragraph?



Tip - Working with Splits
"C:\Program Files (x86)\Vim\vim80\gvim.exe" -O3 *  #Open all files in 3 vertical splits!

:split filename  - split window and load another file; :sp
:vsplit file     - vertical split; :vs
:e filename      - edit another file
:e# toggle files
'A-Z marker between files.

ctrl+w (hjkl) move between windows
 ctrl-w up arrow  - move cursor up a window
 ctrl-w ctrl-w    - move cursor to another window (cycle)
 ctrl-w_          - maximize current window
 ctrl-w=          - make all equal size
 10 ctrl-w+       - increase window size by 10 lines

 :sview file      - same as split, but readonly
 :hide            - close current window
 :only            - keep only this window open
 :ls              - show current buffers
 :b 2             - open buffer #2 in this window

You can use the option -o to open the files in horizontal splits or -O to open vertical splits. The following commands open a window for each file specified:

vim -o *.cpp
vim -O foo bar baz

You can tell Vim the maximum number of windows to open by putting an integer after o or O options, the following example will open at most two windows no matter how many file matches, you will see the first two file specified on the command line, the rest will remain hidden:

vim -o2 *.cpp

See :help -o for all the details.

:args app/views/*.erb | all    #for horizontal splits or

:args app/views/*.erb | vertical all   #for vertical.


Tip - Command Line Mode
First /  to Go in clm search OR : to go in clm commands
 ctrl+p show previous historical search
 ctrl+n next search
ctrl+f

q/ history of search patterns
q: history of commands

Tip - Multiple Searches in vim 
I'm not sure if there's any editor that can do this, we can search for multiple separate words in a single search, here's couple of mechanisms for doing that:
/w1\|w2\|w3
\vw1|w2|w3

If lets say you want many words highlighted then place this in your .vimrc file - really good way to always highlight few keywords you want. (Ref 9,10)
:match Float /weblogic.home\|-Xmx\|-Xms\|weblogic.Name\|config.dir/


Tip - Handle Multiple Files
You can open multilple files simply by vi a.txt b.txt etc. or lets say you did
a grep and found many files as a result you want to go over ,then you can use
something like this to open all at once:
vi $(!!) ##Output of last command
vi $(grep -il "webcat" *.log)
$ find -name testname.c -exec  vi {} +

:n
:N
:args  #shows all files opened with current in brackets.
:rew
:help buffer
Ctrl+g #what file
 :ls              - show current buffers
 :b 2             - open buffer #2 in this window


Tip - Copy and paste to a different file within vim opened files
ma #mark the line
go to other line
y`a #yank from the marker to current line
:ex test.txt #open a new file
p #paste it
:rew #go back to old file

Alternatively:
ma
mb
`a,`byz #y->yank z->buffer
"zp #buffer paste



Tip - Pasting text without messing up formatting

:set paste  #auto indent destroys paste formatting
:set nopaste
set pastetoggle=<F2> # to easily toggle setting
"*p or "+p for pasting from clipboard.

Tip - Vimdiff
Sorry - but I typically use kdiff for better visualization !

Tip -  converting from \r\n to \n only
http://vim.wikia.com/wiki/File_format
:argdo set ff=unix|w

Tip - Highlight cursor
:autocmd CursorMoved * silent! exe printf('match IncSearch /\<%s\>/', expand('<cword>'))
:set hlsearch


Tip - Stop messing with the color and syntax
:syntax off
:set syntax=
syntax off
set nohlsearch
set t_Co=0  #line number color off
set t_Co=0  syntax=strace

Sometimes colors are useful, I typically use zellner which works on most terminals
:colo zellner

Tip - Miscellaneous

gUaw # Uppercase a word
* #search for current word which cursor is on

/echo.*file #Search with pattern
 :%s/^\(Martin\)/Mr \1 Wicks/g  #Group search and replace
4!!awk '{print "New text",$0}' #Appends New text to front of line for 4 lines. you can do this with substitute ^ too, where awk is not needed.
:1,$!awk '{print $1}' | sort | tr [:lower:] [:upper:]  #Just get first column from file sorted and in upper case; replace $1 with $0 to sort have all fields;
:.,$s/^/\t/  #substitue front of line with tabs.
:1,$s/^M//
:s/ \+/\r/g #Break from spaces to newlines

Show special characters
 https://yakking.branchable.com/posts/whitespace-safety/
http://www.shsu.edu/~csc_tjm/fall2000/cs431/shellprog.html
Just to clarify, :set list shows newline ($), :e ++ff=unix shows CR (^M); if you want to see both, :set list then :e ++ff=unix
 or :set binary or vi -b file
 vi -b myfile.txt


Delete

To delete forward up to character 'X' type dtX
To delete forward through character 'X' type dfX
To delete backward up to character 'X' type dTX
To delete backward through character 'X' type dFX



daw #delete a word ; even if you're at middle of word
daw
daW
diw #insert mode
g!/TEXT/d - deletes lines which don't have TEXT in it.. note the !
:g/^$/d - Removes blank lines; Trim lines ; remove duplicate lines  :sort  u ;



Tip - Pad + with spaces - From Ref. 8
f+   #find +
s     #delete char at cursor i.e + and go in insert mode
_+_  #pad + with spaces
;    #repeat find with f
.   #repeat last sequence i.e pad +

 Folding
 http://vimdoc.sourceforge.net/htmldoc/fold.html
 https://www.linux.com/learn/vim-tips-folding-fun
 http://stackoverflow.com/questions/8316139/how-to-set-the-default-to-unfolded-when-you-open-a-file

 zR #Open all folds
 zM #Close all folds
 zi #Toggle.
zf#j creates a fold from the cursor down # lines.
zf/string creates a fold from the cursor to string.
zj moves the cursor to the next fold.
zk moves the cursor to the previous fold.
zo opens a fold at the cursor.
zO opens all folds at the cursor.
zm increases the foldlevel by one.
zr decreases the foldlevel by one.
zd deletes the fold at the cursor.
zE deletes all folds.
[z move to start of open fold.
]z move to end of open fold.

Tip - Good Links for VIM
http://www.vim.org/  #Main site - download from here

http://vimhelp.appspot.com/vim_faq.txt.html
https://www.cs.swarthmore.edu/help/vim/markers.html 
 
 
Tip - Sessions in vim
 :mksession ~/sess.vim
:source ~/sess.vim
gvim -S ~/sess.vim
--save with :xa or :wqa
-might want to set this in .vimrc
set ssop-=options    " do not store global and local values in a session
set ssop-=folds      " do not store folds
 
Tip - Join multiplie lines
J- Joins next lin
[range]j[lines]
%j - joins whole buffer.
%j! - don't replace newlines with spaces..

Tip - Keep portion of text from lines 
%s/\(\[.*, \).*/\1/g
-In \( give pattern to match , then end with \)..then for replace,put that remembered pattern with \1
 
Tip - Working with Macros 
qd     start recording to register d
...     your complex series of commands
q     stop recording
@d     execute your macro
@@     execute your macro again 
In the link above the first command should be  :s/\s\+/': ' (then press Enter)
 
References Used
  1. http://vimregex.com/
  2. http://www.ibm.com/developerworks/aix/library/au-vitips.html
  3. http://stackoverflow.com/questions/1551231/highlight-variable-under-cursor-in-vim-like-in-netbeans
  4. http://alvinalexander.com/linux/vi-vim-editor-color-scheme-colorscheme
  5. http://vim.wikia.com/wiki/How_to_turn_off_all_colors
  6. http://vimcasts.org/episodes/refining-search-patterns-with-the-command-line-window/
  7. http://vim.wikia.com/wiki/A_better_Vimdiff_Git_mergetool#Vimdiff_for_three-way_merges
  8. Practical Vim, 2nd Edition By Drew Neil
  9. http://stackoverflow.com/questions/704434/is-there-any-way-to-highlight-multiple-searches-in-gvim
  10. http://stackoverflow.com/questions/4162664/vim-highlight-a-list-of-words#comment4494852_4162735 
  11.  https://missing.csail.mit.edu/2020/editors/
  12. https://stackoverflow.com/questions/1642611/how-to-save-and-restore-multiple-different-sessions-in-vim


Creating a new Oracle Linux Virtualbox Virtual Machine for personal use

I'll take you though creating an Oracle Linux 6.5 VM on a Windows PC. This was the latest Linux available at the time.
Creating a VM gives you all the benefits from my earlier post. Also, you can build your own image from latest linux builds and configure it the way you want to.

Install latest VirtualBox and its guest additions on your local PC from its website.
Download the OS Image Oracle Linux 6.5 from edelivery site https://edelivery.oracle.com/linux/ . It will be in an ISO File.

In Oracle VM VirtualBox Manager, File -> Preferences -> General -> Choose Correct Default Machine Folder. Example, E:\OVM , especially if you’re creating VMs on a different disk.

In Oracle VM VirtualBox Manager ,choose New

Create a NEW machine, enter a name you want to give


Allocate enough memory to VM as needed. Typically 2GB should be good enough for many tasks in Linux, but if you want to develop in NetBeans for example you should give around 4GB.

 Choose do not add a virtual hard drive yet. We’ll add those later. Hit Create. On the listed newly created VM, select it and choose Settings, to modify its settings.

In Settings go to Storage options first and near Controller:SATA hit the icon to add disk.

Choose VDI file type. This format has benefit of compacting files and resizing.

 On next screen choose “Dynamically Allocated” physical hard drive.
Next,  choose file location and size. Give a good name (name_DATA) and location and select around 100 GB or as seen necessary for your work.
Add another disk (name_SWAP) for swap partition on linux. Allocate 8 gb file. I chose dynamically allocated for swap, but if performance is of concern you can choose fixed type for better performance. You’d have to wait for the big file to be created though.
Next In  Settings -> storage ->  Controller:IDE , CD/DVD Drive. Load up the ISO by clicking the cd icon next to drop down.

Hit OK. Start the VM by double clicking or hitting Start.
Virtual Box starts up the Linux installation process.

Hit Ok on this message, once you get Linux fully installed it won’t affect you.
  Follow the upcoming menus
  Choose ntp time server
  Choose Create Custom Layout  option, instead of use all space option, so that you can layout swap in other 6gb partition.


Create standard partition on the large 30gb hard drive, give its mount point as /


 On other drive choose swap as the file system type and click fill to maximum allowable size.

Choose Software Development Workstation installation option; Note that basic server option doesn't give gui and is useful for server class systems; desktop option also had issues in installing VB guest additions.


Create group dba, user oracle by using advance user creation option.

  Reboot your VM
You should Install guest additions as it gets rid of the mouse locking , help resize your window automatically .
Once the Linux VM is up, do an  yum update ; repeat if needed. You’re now all set to install your favorite software on it.

Please let me know if you found this useful, It took me some time to figure this out even with help of a good reference as mentioned below..

Author: Dhiraj Soni
Date: Feb 4, 2014

References:
VirtualBox - Creating a Base OEL Image, by Jim Fisher   Nov 4, 2010

Thursday, March 2, 2017

VirtualBox - a virtualization technology


One of the technologies I find hard to live without is the virtualization technology. On daily basis I utilize Linux OS virtual machine(VM) running on my Windows Laptop. If you're not using this technology yet, perhaps this article can help you look into utilizing this.

Virtualization is seen as one of the enabling technologies of this generation. It enables much better utilization of the huge processing power of machines now available. Still, this was largely seen as domain of enterprise computing not at a laptop level. With advent of laptops of 4GB+ ram, it became more feasible to utilize virtualization at desktop/laptop levels. One such enabling technology is using open source VirtualBox (VB) virtualization (https://www.virtualbox.org/). This is comparable to VMWare if you're familiar to that.

With VB a key benefit is you get a self contained virtualized OS VM platform which you can move around with ease on any other PC you have. The virtual OS could be of many flavors like Windows, Solaris, Linux and Mac OS X. You can install any software on that VM and work with it like you do on a native OS. 

You can possibly work with multiple VMs on a single laptop, and you can also relocate the full working environment to other laptops with ease. You don't have to go though painful process of reinstalling your software on OS on other laptop where you're moving to.

Another benefit is to have a snapshot of the VM available, this is a point in time of the current state of your OS and you can revert back to this state later. This allows you to install software and if it doesn't work then you can revert back your VM state of the last snapshot. This saves a lot of time and effort if something goes wrong.

These days many vendors like Oracle give out pre-built VMs with their software installed on it. So you save lot of time by just reusing their VMs as your starting point for your development work. Few pre-built developer VMs are available here at oracle site.

VirtualBox’s many other benefits can be found in its very useful documentation. You can read through this manual to get a good understanding on how to go about setting up and using this technology.