4 Hot skills for Linux pros in 2017

ONE OF THE PROBLEMS with becoming a Linux expert is the definition is constantly changing. When I started in the Linux world, to be considered a Linux professional, you had to be able to compile your own kernel. Heck, if you wanted to use Linux on a laptop, you had to compile a custom kernel to even be a user. These days, compiling your own kernel is usually a waste of time. That’s not to say it isn’t important, but in the open source world we build on the successes of others, and Linux distributions provide us with kernels that work well. Although not always that drastic, the demands on IT professionals change every year.

Here are four vital skills for the Linux pro in 2017:

1. Security
I’m not talking about security experts or security consultants. With connected devices infiltrating every aspect of our lives, we need to be security conscious in every decision we make.
When we activate any system at work, home, or in our pockets, we should consider the security issues they might represent. And because items like Internet-enabled toasters aren’t likely to get timely firmware upgrades, we need to design the rest of our systems around the idea of mundane devices getting compromised. More than ever before, we need to think about attacks coming from inside our firewalls. Don’t let your file server get hacked by your blender!

2. DevOps
DevOps is no longer a new concept. For the past two or three years, we’ve been encouraging folks to learn about DevOps so they can succeed in the workforce. That was good advice, but it doesn’t mean we should rely completely on automation tools to do our jobs. Chef, Puppet, Ansible, Salt Stack, and similar tools are wonderful, but we need to understand what’s happening behind the scenes so when something inevitably goes wrong, we know how to fix it.
With DevOps’ programmatic approach to computing, we still need people who can  maintain, fix, and understand the systems functioning beneath the layer of code. Without Linux experts, cloud computing is a scary place to live, even if that cloud is in your own server room.

3. Development
As a system administrator for 3 years, I never had the time to learn programming. Any development skills I had were basically scripting that helped me do my job faster. Those days are over. While we need to have system administration skills in a DevOps world, we also need system administrators to have programming skills.
If you’re a crusty old sysadmin like me, you’ve probably adopted DevOps and use it on a daily basis. If you truly want to excel, however, you need to learn how to solve problems programmatically and not think of Chef or Puppet code only as configuration files. Every IT professional needs to have at least a grasp of programming concepts, because every aspect of IT is getting abstracted at least somewhat by DevOps code.

4. Soft skills
Often the last thing we think about while preparing for a career are so-called soft skills—social and communication skills—and yet they are probably skills most likely to determine your success. Whether you’re looking for a new job, or trying to adjust to the changing landscape of your current career, soft skills are vital.
The lines dividing the various areas of IT are blending, and the ability to communicate well makes those blurred lines an advantage instead of a stumbling block. We live in a world in which developers are spinning up servers, and operations teams are writing Ruby code to maintain server farms. These are bold new ideas in IT, and without people able to communicate between disciplines, the workplace becomes hostile quickly. Plus, IT folks have always needed to communicate effectively with people in other areas of business. If anything, that need is greater now than ever.

As you plan for 2017, what skills are you adding to your skill set?

Do comment…!!!

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

Advertisements

Should MySQL and Web Server share the same box ?

This is interesting question which I thought it would be good to write about. There are obviously benefits and drawbacks for each of methods.

Smaller applications usually start with single server which has both MySQL and Web server on it. It is not usually the question but once application growths larger and you need to have multiple servers you may decide ether to grow system in MySQL+Apache pairs or split MySQL And Web Server and place them on different boxes.

Generally using separate boxes for MySQL and Web Servers is rather good practice.

It is more secure – Compromising your web server does not directly give access to your database, even though most applications have enough database access permissions to be allow intruder to trash/dump data.

It is easier to analyze – Troubleshooting bottlenecks on shared boxes is more complicated compared to systems running only MySQL or only Web server. In this case you already know who is troublemaker by simply looking at system wide stats.

Easier to maintain – Same thing if box happens to run multiple things it is harder to maintain. I would not call the difference significant in this case though.

Easier to balance – Lets say you have Web application and just added some new feature, ie chat application which increases load on your web server but does not really affect database part of load. If you can operate database farm and web server farms separately you can simply increase number of web servers.

It is less expensive – You typically want database boxes to be secure, using good hardware with ECC memory to avoid database corruption, use RAID to avoid loosing database with any hard drive loss etc. Database boxes also generally require more monitoring and maintenance such as backups so you end up using some serious hardware for this boxes to keep their number manageable. With Web boxes it is different – you’re quite OK using crappy hardware for them as all you need is CPU power. If box starts to misbehave it is easy to shut it down without affecting site operations. Also you rarely would have data corruption due to web boxes memory failure, more likely you’ll have web server crashes and this sort of things. You can ether clone web servers from template hard drive or even have them disk less booting by NFS.

So if using dedicated boxes is so great why to think about sharing MySQL and Web server at all ? Well mostly it is for cheap guys.
In many applications you will find database servers to be IO bound so CPUs are doing virtually nothing and you’re wasting resources. This is the reason for some cheap environments to have Web servers also on database boxes, might be only handling partial load etc.

I would however only use it in last resort – placing some data crunching scripts on database server is often better use of their free CPU time.

Second thing you may feel bad about it is Web Servers memory. Getting certain amount of memory is pretty cheap, ie 4GB of memory per box costs very close to 2GB, while jump from 16GB to 32GB may be much more expensive (even in price by GB).
So you can get Web boxes with relatively plenty of memory cheap but unless you’re running 500 Apache children with mod_P (php,perl,python) per box (which is probably bad idea anyway).

The good use for such extra memory is probably caching – Web page caching, if you do not have separate layer for it, local memory or cacheing type of caching (depending on your application needs) is very good idea.

One more benefit of local access to MySQL is latency. This was problem many years ago with 10Mbit network but with 1Gbit networks being commodity these days you should not worry too much about it, unless you have each page generated by 1000+ queries, which is bad idea already.

One case I should mention when shared MySQL and Web server makes sense is Web Services architecture when you can have certain boxes providing you with some simple “Services” – these could be small enough to be single shared box (or pair of shared boxes for HA). In such cases I would think about Web Server mainly being provider of different protocol to access your data – it is typically simple and would not require much of CPU and other resources itself.

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

 

How To Log Into a VPS with PuTTY (Windows Users)

Introduction

You finally own a VPS (Virtual Private Server), but you are probably still wondering and saying: “How am I going to connect to my sever, since I have no cpanel to type my username and password into?”. Let me assure you, with Debian (what we’ll use in this tutorial) or any other Linux distribution, you could achieve and do anything a cpanel or any web panel can do and EVEN MORE.

First Time Connecting to your VPS

Your VPS might be located thousands of miles away from you. However, with the help of a couple of programs, you can connect to it as if it were in front of you. And most importantly, all of these programs allow you to connect safely to your server through what is called ‘SSH’.

What is SSH?

SSH (Secure Shell) is a network protocol used for secure data communication between a server and a client (You) to perform (for example: command-line login and authentication, remote command execution, and even data transfer). So in order to keep the communication between you and a server secure from the preying eyes of hackers, there are programs that implement SSH protocols mainly by using strong encryption methods to help you achieve that (Figure 1).

1

Figure 1: Basic Concepts of SSH Protocol

OpenSSH and PuTTY

To establish communication between a client and a server, you must have SSH program on each communicating end. Hence come OpenSSH and PuTTY, which are only two SSH programs from several others. OpenSSH is the most popular and most widely used SSH program that Debian comes shipped with. PuTTY is the most popular SSH program in Windows OS. In this tutorial, I will explain how to use them correctly in order to communicate securely with your server.

Installing and Configuring PuTTY in Windows

Since OpenSSH is already installed on our server, you only need to install PuTTY before you can connect to your server. Go to PuTTY’s official download page from here and download the installer file which looks like [putty-x.xx-installer.exe]. After downloading it, install as you would install any Windows program; just make sure to install it only for the current user, especially if your PC is used by many users. Once the install is finished, launch PuTTY and a configuration window will appear to you (Figure 2).

2

Figure 2: PuTTY Configuration Window

Now, just follow the following steps in order to connect to your server for the first time:
1. In the **Host Name (or IP address)** field, enter your Server’s IP address.

2. Just make sure that the Port field shows number 22; since it’s the default port number for SSH protocol, and also the Connection type set to SSH as in (Figure 2).

3. In the Saved Sessions field, write a name for your session and then hit SAVE; this will save all the configurations we did earlier; so that the next time you launch PuTTY, you wouldn’t have to enter your server’s configuration and your PuTTY preferences all over again.

4. Finally you are ready to connect to your server, either by selecting the session name and then clicking the Open button at the bottom or simply double-clicking the session name that you have saved earlier.

Server-Client authentication

First: Server authentication

3

Figure 3: PuTTY Security Alert

After initiating the connection to your server as we discussed earlier, you’ll notice that the configuration window disappeared and instead a black terminal window has appeared but with a ‘Security Alert’ (Figure 3).

Don’t freak out, this alert is expected to appear ONLY the first time you connect to a server that you’ve never connected to before. To make it simple for you to understand why this scary alert window appeared, I’m going to use a simple analogy: say you drove a fancy car to a 5-star hotel, then a valet asks for your car’s key in order to park it. The question is: Do you trust this guy who is asking for your fancy car’s keys? The answer to this question will determine your final decision. Your answer is going to be based (subconsciously) at least on:

  • The guy’s clothing.
  • The badge he is carrying and/or the hotel’s logo or name on his uniform.

Exactly the same happens when your client’s SSH program (PuTTY) connects for the first time to your server’s SSH program (OpenSSH). Now you, as a client, have a very sensitive information which is your login credentials, i.e. your Debian server’s account username and password. So as you wouldn’t give your fancy car’s key to a total stranger, neither would PuTTY to an unknown sever.

4

Figure 4: 2 steps of Server client Authentication

Figure (4) above explains concisely the operations taking place in the background during sever authentication and establishment of a secure connection between the two ends:

1. PuTTY contacts the OpenSSH.

2. OpenSSH identifies itself to the PuTTY by sending it a Host key and some other parameters.

3. PuTTY, in turn, searches through its known hosts database to see whether the Host key that OpenSSH has sent in step 2 exits or not.

4. If NOT, then before terminating the current session, the security alert window (Figure 3) would appear; if YES, then proceed to step 5.

  • So, a Host key to PuTTY is basically what a valet’s uniform and badge is to a car owner. It is simply a unique fingerprint to your server’s SSH program (OpenSSH) that helps PuTTY in future sessions to recognize it by.
  • Back again to the ‘Security Alert’ window in (Figure 3), assuming you have entered your server’s IP address correctly, it would be safe now to click ‘Yes’. What that basically does is tell PuTTY to save your server’s Host key in its known hosts database in windows registry to be used for future authentications.

5. Finally, server authentication is completed and secure connection is established.

Second: Client Authentication

Since you are finally connected securely to your sever, it is safe now to send your login credentials through the SSH encrypted connection (Figure 4). The picture below shows a terminal command line where you are going to enter your username in the line that says login as:, then hit Enter, then a new line will appear asking for your password, type it and hit Enter.

5

Figure 5: The beginning of Client authentication Process

You will know that the client authentication process was successful when the terminal shows you what appears in (Figure 5). This is basically brief information about the Linux distribution that is installed on your server, information about the last time you logged in, and the last line is where all the magic happens and it’s called the ‘Command or Prompt line’, and its structure is similar to the following:

 username@hostname:~#

The first part indicates the username that you are currently logged in as then the host name, separated by the ‘@’ symbol, followed by your current directory (in this example ~, which refers to current user home directory) and the hash sign indicates the end of command line.

ATTENTION!

For every Linux distribution, the command line might differ slightly, but basically they all have the same structure.

Conclusion

Learning is a slow and progressive process and I hope that you have grasped the basic concepts behind remote connection to a server and why security matters so much in the ever crowded insecure internet world and how SSH helps you maintain that. So remember that your server is like your home, you always have to protect it from unwanted visitors.

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

Install GIMP Image Editor On Ubuntu 16.04

GIMP is a cross-platform image editor available for GNU/Linux, OS X and Windows.

GIMP provides the tools needed for high quality image manipulation. From retouching to restoring to creative composites, the only limit is your imagination. GIMP is used for producing icons, graphical design elements, and art for user interface components and mockups. GIMP provides top-notch color management features to ensure high-fidelity color reproduction across digital and printed media. It is best used in workflows involving other free software such as Scribus, Inkscape, and SwatchBooker.

Install GIMP

Ubuntu, Mint, or Debian users can run apt-get install gimp to install GIMP. Ubuntu users may also install GIMP from Software Centre, this includes recent GIMP versions from PPAs.

Run the following commands in Terminal to install GIMP 2.9.3 on Ubuntu Systems:

Install GIMP 2.9.3

root@ubuntu-DL-960GM-GS3-FX:/home/ubuntu# sudo add-apt-repository ppa:otto-kesselgulasch/gimp-edge

gimp1

gimp2
root@ubuntu-DL-960GM-GS3-FX:/home/ubuntu# sudo apt-get update

gimp3

gimp4
root@ubuntu-DL-960GM-GS3-FX:/home/ubuntu# sudo apt-get install gimp

gimp5

gimp6

Remove GIMP 2.9.3

Run the following commands in Terminal to uninstall and remove GIMP 2.9.3 from Ubuntu Systems:

root@ubuntu-DL-960GM-GS3-FX:/home/ubuntu# sudo apt-get install ppa-purge

root@ubuntu-DL-960GM-GS3-FX:/home/ubuntu# sudo ppa-purge ppa:otto-kesselgulasch/gimp

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

How to Create a SWAP space in an AWS Micro Instance

aws-logo

If you are using a Micro Instance in AWS then you can find there is no swap space with it. But, it always requires a swap space if you are running applications that require a large amount of memory. The combined sizes of the physical memory and the swap space is the amount of virtual memory available. As per the old rule the swap partition should be twice the size of the physical memory. As per Red Hat Recommendation- “System with 4GB of ram or less require a minimum of 2GB of swap space”. In this article I will show you how to find out your swap space and how to create it.

First we will determine that how much swap space we have.

To check the free available physical memory as well as swap memory

[root@ip-10-0-1-41 admin] # free -m

To check the disk space & partition layout

[root@ip-10-0-1-41 admin] # df -h

You can verify the swap space using the below commands:

[root@ip-10-0-1-41 admin] # swapon -s

OR

[root@ip-10-0-1-41 admin] # cat /proc/swaps

We can create swap space using two methods with dd, mkswap and swapon commands.

First Method :-

Create a swap file

[root@ip-10-0-1-41 admin] # dd if=/dev/zero of=/swap bs=1M count=1024

Make the file as a swap file

[root@ip-10-0-1-41 admin] # mkswap -f /swap

Enable the newly created swap file

[root@ip-10-0-1-41 admin] # swapon /swap

Second Method :-

If you have additional free hard disk then you can use that as your swap partition. Assuming your free disk name as “/dev/sdf”

[root@ip-10-0-1-41 admin] # mkswap -f /dev/sdf

Enable the swap partition for usage

[root@ip-10-0-1-41 admin] # swapon /dev/sdf

To Make the swap space available after boot you have to edit the /etc/fstab file.

[root@ip-10-0-1-41 admin] # vi /etc/fstab

  • /swap                  swap                    swap    defaults        0 0

OR

  • /dev/sdf               swap                    swap    defaults        0 0

1

Now you can verify the swap space again using the earlier discussed command.

I have added my EC2 Instance to my SAAS Monitoring tool. You can see the screenshot of the added Monitor to verify that the swap memory is also being monitored.

capture1

 

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

 

 

MBR and GPT partition tables

In this post we will be discussing about the following things.

  • 1.What is MBR partition table?
  • 2.What were the demerits of MBR partition structure?
  • 3.What is GPT Partition table?
  • 4.What are the plus points of GPT?
  • 5.What tools in Linux can be used to create GPT partition structure?

With the increasing demand for large size partitions, the partitioning and file systems are becoming more complex and advanced.

There are two mainly used disk partition styles.They are as follows :-

1.MBR (Master Boot Record) partitioning style

2.GPT  (GUID Partition Table)

Both the above mentioned schemes are used to identify the location of the partition on the hard disk.the type of partition used is decided when the disk is initiliazed.

the above shown diagram shows a disk initialization dialog in windows, when a new disk is added to the machine. when you initialize the disk these days you are presented with exactly the same options as shown in the above figure.

Master Boot Record (MBR) or Globally Unique Identifier Partition Table (GPT.).Previously there was only one option MBR.

MBR Partitioning style:

In this style the MBR resides in the first 512 bytes of the hard disk. This 512 bytes contains two things in MBR partitioning scheme.

1. 440 bytes for boot loader(Grub, or windows boot loader)

2. the remaining space is used for the partition table, ie: there can only be 4 primary partition in an MBR partitioning scheme.this remaining area also has the partition location information.

This can be done by two methods

  • CHS method: in this method a partition is located through cylinder,head,sector method. this is mentioned by a 24 bit number, in this case it can only mention disk at the max of 8gb.
  • LBA method: LBA stands for Logical Block Addressing, in this each and every sector is numbered serially resulting easy addressing of spaces upto 2TB.

Demerits of MBR partitioning scheme:

  1. It only allows four primary partitions.
  2. The workaround for creating more than four partition by making one of them extended(containing logical partitions inside that) this also leads to problems sometimes as some operating systems can only boot from Primary partitions.
  3. The CHS implementation in MBR partition table can lead to problems because the Cylinder,Head,Sector geometry of the disk can change.
  4. Even the LBA scheme implemented puts a 2tb limitation.
  5. Logical partitions are stored in Linked List data structure over the extended partition, and can easily fail, which intern make can make some logical partitions inaccessible.

GPT Partition Style:

Now the MBR type partition table keep the partition table info such as information about the four primary partition in the first 512 bytes of the hard disk. In case of GPT the first sector is kept for “protective MBR” so that, old bios based computers can also boot from GPT partition table.

The GPT partition information starts from the second block or call it LBA 1

Note: LBA is nothing but the sectors numbered in such a way that it can be easily addressed, like LBA 0 is for MBR,LBA 1 for GPT header info etc.

And most of the time LBA 2 contains GPT partition array entry.

The partition array entry typically reserves 128 bytes for each partition entry. Now for a disk of 512-byte sectors the partition array entry can reach up to 16,384 bytes. And in most cases the 34’th sector or call it block is the first usable sector in the disk.

Merits of GPT partition Style:

1.GPT supports up to 128 partitions so there is no need for extended of logical partitions.

2.GPT addresses 64 bit number LBA scheme, so it can address up to 8 ZiB size partitions.

3.GPT structure have primary and secondary header, so recovery is more promising that MBR

4.GPT provides 1 byte partition code, but GPT give you 16byte GUID value to uniquely identify the partition.

Tools for Creating GPT partitions:

Fdisk cannot create a GPT partition so we need to use some other tool to create it.

1.Parted can be used to create GPT partition table, and partitions of more than 2tb size.

2.Gdisk is also a nice tool to create GPT partition table.

 

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!

Viewing Linux Logs from the Command Line

logs_main

At some point in your career as a Linux administrator, you are going to have to view log files. After all, they are there for one very important reason…to help you troubleshoot an issue. In fact, every seasoned administrator will immediately tell you that the first thing to be done, when a problem arises, is to view the logs.

And there are plenty of logs to be found: logs for the system, logs for the kernel, for package managers, for Xorg, for the boot process, for Apache, for MySQL… For nearly anything you can think of, there is a log file.

Most log files can be found in one convenient location: /var/log. These are all system and service logs, those which you will lean on heavily when there is an issue with your operating system or one of the major services. For desktop app-specific issues, log files will be written to different locations (e.g., Thunderbird writes crash reports to ‘~/.thunderbird/Crash Reports’). Where a desktop application will write logs will depend upon the developer and if the app allows for custom log configuration.

We are going to be focus on system logs, as that is where the heart of Linux troubleshooting lies. And the key issue here is, how do you view those log files?

Fortunately there are numerous ways in which you can view your system logs, all quite simply executed from the command line.

/var/log

This is such a crucial folder on your Linux systems. Open up a terminal window and issue the command cd /var/log. Now issue the command ls and you will see the logs housed within this directory (Figure 1).

logs_a

Now, let’s take a peek into one of those logs.

Viewing logs with less

One of the most important logs contained within /var/log is syslog. This particular log file logs everything except auth-related messages. Say you want to view the contents of that particular log file. To do that, you could quickly issue the command less /var/log/syslog. This command will open the syslog log file to the top. You can then use the arrow keys to scroll down one line at a time, the spacebar to scroll down one page at a time, or the mouse wheel to easily scroll through the file.

The one problem with this method is that syslog can grow fairly large; and, considering what you’re looking for will most likely be at or near the bottom, you might not want to spend the time scrolling line or page at a time to reach that end. Will syslog open in the less command, you could also hit the [Shift]+[g] combination to immediately go to the end of the log file. The end will be denoted by (END). You can then scroll up with the arrow keys or the scroll wheel to find exactly what you want.

This, of course, isn’t terribly efficient.

Viewing logs with dmesg

The dmesg command prints the kernel ring buffer. By default, the command will display all messages from the kernel ring buffer. From the terminal window, issue the command dmesg and the entire kernel ring buffer will print out (Figure 2).

logs_b

Fortunately, there is a built-in control mechanism that allows you to print out only certain facilities (such as daemon).

Say you want to view log entries for the user facility. To do this, issue the command dmesg –facility=user. If anything has been logged to that facility, it will print out.

Unlike the less command, issuing dmesg will display the full contents of the log and send you to the end of the file. You can always use your scroll wheel to browse through the buffer of your terminal window (if applicable). Instead, you’ll want to pipe the output of dmesg to the less command like so:

dmesg | less

The above command will print out the contents of dmesg and allow you to scroll through the output just as you did viewing a standard log with the less command.

Viewing logs with tail

The tail command is probably one of the single most handy tools you have at your disposal for the viewing of log files. What tail does is output the last part of files. So, if you issue the command tail /var/log/syslog, it will print out only the last few lines of the syslog file.

But wait, the fun doesn’t end there. The tail command has a very important trick up its sleeve, by way of the -f option. When you issue the command tail -f /var/log/syslog, tail will continue watching the log file and print out the next line written to the file. This means you can follow what is written to syslog, as it happens, within your terminal window (Figure 3).
logs_c

Using tail in this manner is invaluable for troubleshooting issues.

To escape the tail command (when following a file), hit the [Ctrl]+[x] combination.

You can also instruct tail to only follow a specific amount of lines. Say you only want to view the last five lines written to syslog; for that you could issue the command:

tail -f -n 5 /var/log/syslog

The above command would follow input to syslog and only print out the most recent five lines. As soon as a new line is written to syslog, it would remove the oldest from the top. This is a great way to make the process of following a log file even easier. I strongly recommend not using this to view anything less than four or five lines, as you’ll wind up getting input cut off and won’t get the full details of the entry.

There are other tools

You’ll find plenty of other commands (and even a few decent GUI tools) to enable the viewing of log files. Look to more, grep, head, cat, multitail, and System Log Viewer to aid you in your quest to troubleshooting systems via log files.

Don’t forget to like & share this post on social networks!!! I will keep on updating this blog. Please do follow!!!