Git and word documents

If you got word documents in the git repo and tired of seeing binary diff message from git, do the following setups to fix it.

$ git diff
diff --git a/hello.docx b/hello.docx
index bc73959bc592..122761966158 100644
Binary files a/hello.docx and b/hello.docx differ

To see diff of what has been changed in the word document, do following:

  • Install docx2txt. You can download it from http://docx2txt.sourceforge.net. Follow the instructions in the INSTALL file to put it somewhere your shell can find it.
  • Create or append file ~/.config/git/attributes to have

*.docx diff=word

  • Edit .gitconfig to have following commands

[diff "word"]
textconv = ~/git_docx2txt.sh

[alias]
wdiff = diff --word-diff=color --unified=1

  • Create ~/git_docx2txt.sh with following text

!/bin/bash

docx2txt.pl "$1" -

  • chmod a+x ~/git_docx2txt.sh
  • You will have

$ git diff
diff --git a/hello.docx b/hello.docx
index bc73959bc592..122761966158 100644
--- a/hello.docx
+++ b/hello.docx
@@ -1 +1 @@
-Hello world
+Hello world: View word diff in git.

$ git wdiff hello.docx
diff --git a/hello.docx b/hello.docx
index bc73959bc592..122761966158 100644
--- a/hello.docx
+++ b/hello.docx
@@ -1 +1 @@
Hello worldworld: View word diff in git.

Happy diff view.

Setting up cppcheck

To install pre-built cppcheck: http://cppcheck.sourceforge.net/

sudo apt-get update
sudo apt-get install cppcheck

In case, you want to use latest cppcheck, download and build it from the source code

wget https://github.com/danmar/cppcheck/archive/2.1.tar.gz
tar zxvf 2.1.tar.gz
cd cppcheck-2.1
mkdir build
cd build
cmake ..

P.S: In case you are interested for cppcheck GUI use cmake -DBUILD_GUI=ON ..

make
sudo make install

Once installation is complete.

which cppcheck
/usr/local/bin/cppcheck

cppcheck --version
Cppcheck 2.1

Build your cpp project with cmake and pass on an extra parameter -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE to cmake. It will produce  compile_commands.json file in your build folder for the project.

Execute cppcheck -p </path/to/folder/having/compile_command.json>.

To use it in VSCode add following to the tasks.json

{
    "label": "Run cppCheck",
    "command": "/usr/local/bin/cppcheck",
    "args": [
       "--project=${workspaceRoot}/build/compile_commands.json"
    ],
    "problemMatcher": "$gcc",
}

Ref: https://gist.github.com/aakbar5/268a2072138345893b7f82590dcc3d26

Linux kernel & VS Code

If you are using VSCode for Linux kernel development, following setup will be useful:

  1. Clone https://github.com/aakbar5/handy-kernel_modules. It contains the
    • Makefile: to build kernel modules
    • .devcontainer: to setup vscode for remote development inside the Docker container
    • .vscode: From tasks build and run Qemu.
  2. Start vscode
  3. Open the folder having linux kernel development project
  4. Select: Remote-Containers: Open Folder in Container… from the popup menu.
  5. Select the folder selected in #2
  6. Keep an eye on the vscode status bar while it is showing Opening Remote…
  7. Once vscode is setup, your project will be ready for development in container environment.
  8. Press CTRL+P for Tasks: Run Task
  9. Select kmod – build task to build a kernel module setup in Makefile.
  10. Press CTRL+P for Tasks: Run Task
  11. Select Generate compile_commands.json from the list of tasks.
  12. Press CTRL+P for Tasks: Run Task
  13. Select Run QEMU and wait for QEMU to be ready for use
  14. Press CTRL+P for Tasks: Run Task and select kmod – load

    • This command will copy built kernel module to QEMU.
  15. Get back to QEMU shell in vscode terminal and use kernel module commands to test your new kernel module.
  16. Once you are done with development, use Dev Container: Close Remote Connection option.

kernel_in_vscode

dup2 usage

// A simple example of how dup2 can be used
const char* msg = "Text written by standard file descriptor\n";
int fd = open("dup2_test.txt", O_RDWR | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR);
write(fd, msg, strlen(msg));
/* Map stdout to our file */
dup2(fd, STDOUT_FILENO);
printf("Text written by dup @ stdout file descriptor\n");
close(fd);

view raw
dup2_usage.c
hosted with ❤ by GitHub

Setting up include-what-you-need (IWYU)

Use following instructions to install IWYU (https://include-what-you-use.org/) on Ubuntu 18.04.3 LTS:

sudo apt-get update
sudo apt-get install -y build-essential cmake git zlib1g-dev libncurses5-dev llvm-6.0-dev libclang-6.0-dev libclang-6.0-dev clang-6.0
git clone https://github.com/include-what-you-use/include-what-you-use.git iwyu.git
cd iwyu.git
git checkout clang_6.0
mkdir -p build
cd build
cmake -DIWYU_LLVM_ROOT_PATH=/usr/lib/llvm-6.0 ..
make 
make install

Once installation is complete, you will find `include-what-you-need`

# include-what-you-use --version
include-what-you-use 0.10 (git:a1878c4) based on clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)

To execute include-what-you-need:

Build your cpp project with cmake and pass on an extra parameter -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE to cmake. It will produce  `compile_commands.json` file in your build folder for the project.

Execute iwyu_tool.py -p </path/to/folder/having/compile_command.json>. It may take time depending upon the code size.

fix_includes  can be used to apply recommended changes.

To use it in VSCode add following to the tasks.json

{
    "label": "Run iwyu",
    "command": "iwyu_tool.py",
    "args": [
        "-p",
        "${workspaceRoot}/build"
    ],
    "presentation": {
        "echo": true,
        "reveal": "always",
        "focus": true,
        "panel": "shared",
        "showReuseMessage": true,
        "clear": false
    },
    "problemMatcher": "$gcc"
}

Ref: https://gist.github.com/aakbar5/268a2072138345893b7f82590dcc3d26

VisualStudio code & compile_commands.json

If you are working with C/C++ in VSCode, you will find that VSCode keeps on showing

"${workspaceFolder}/build/compile_commands.json" could not be found. 'includePath' from c_cpp_properties.json will be used instead.

This message is generated by vscode-cpptools. Although it is an optional file however vscode-cpptools keeps on showing error message that it is missing.

This message can easily be fixed if you are using CMake. To do so simply add following to your CMake configuration file.

# Generate compile_commands.json
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

Setup: FTP server

# Setup FTP server
# Instructions are tested on: Ubuntu 18.04.3 LTS
apt install -y atftpd
sudo gedit /etc/default/atftpd
# – set USE_INETD=FALSE
# – Optional: To change directory to host FTP contents, change /srv/tftp in OPTIONS line
# – Optional: To have maximum log: change –verbose=7 and append –logfile /var/log/atftpd.log
sudo mkdir -p /srv/tftp
sudo chmod -R 777 /srv/tftp
sudo chown -R nobody /srv/tftp
# Vanilla contents
# USE_INETD=true
# # OPTIONS below are used only with init script
# OPTIONS="–tftpd-timeout 300 –retry-timeout 5 –mcast-port 1758 –mcast-addr 239.239.239.0-255 –mcast-ttl 1 –maxthread 100 –verbose=5 /srv/tftp"
# After changes
# USE_INETD=false
# # OPTIONS below are used only with init script
# OPTIONS="–tftpd-timeout 300 –retry-timeout 5 –mcast-port 1758 –mcast-addr 239.239.239.0-255 –mcast-ttl 1 –maxthread 100 –verbose=7 –logfile /var/log/atftpd.log ~/tftp"
# Run the service
sudo invoke-rc.d atftpd start
# Verify atftpd is running
netstat -lnp | grep ":69 "
# Use atftp FTP client to verify atftpd
apt install -y atftp
atftp localhost
# Make sure /etc/hosts is having following mapping otherwise localhost above command will not work
# 127.0.0.1 localhost
# Download an file from FTP server
get file_name_which_is_hosted_on_ftp

rename git branches

Let’s rename a git branch locally & remotely.

Let’s assume that you got git repository with multiple branches and you want to mark a branch as a master whereas existing master to something else.

  • Make note of the branch names and their respective remote
git branch -vv
  • Goto server repo and switch default branch
git symbolic-ref HEAD refs/heads/new-master
  • Goto local repo, checkout master
git checkout master
  • Rename it locally
git branch -mv master something
  • Remove remote branch
git push origin --delete master
  • Set new remote
git branch --unset-upstream
git push --set-upstream origin something
  • Goto server repo and switch default branch
git symbolic-ref HEAD refs/heads/something
  • Switch to branch which is going to be new master
git checkout new-master
  • Rename this branch
git branch -mv new-master master
  • Remove remote branch
git push origin --delete new-master
  • Set new remote
git branch --unset-upstream
git push --set-upstream origin master
  • Goto server repo and switch default branch to master
git symbolic-ref HEAD refs/heads/master

Tensorflow grappler

Tensorflow runtime got a module grappler which does optimizations once model is loaded into module. Tensorflow session parameter can be used to control this optimization applied by grappler.

// Tensorflow session
tensorflow::SessionOptions session_opts;
tensorflow::OptimizerOptions* options = session_opts.config.mutable_graph_options()->mutable_optimizer_options();
// Disable optimizations
options->set_opt_level(tensorflow::OptimizerOptions::L0);
// Grappler related optimizations
auto config = session_opts.config.mutable_graph_options()->mutable_rewrite_options();
config->set_layout_optimizer(tensorflow::RewriterConfig::OFF);
config->set_disable_model_pruning(true);
config->set_function_optimization(tensorflow::RewriterConfig::OFF);
config->set_arithmetic_optimization(tensorflow::RewriterConfig::OFF);
config->set_loop_optimization(tensorflow::RewriterConfig::OFF);
config->set_dependency_optimization(tensorflow::RewriterConfig::OFF);
config->set_constant_folding(tensorflow::RewriterConfig::OFF);
config->set_memory_optimization(tensorflow::RewriterConfig::NO_MEM_OPT);
config->set_shape_optimization(tensorflow::RewriterConfig::OFF);
config->set_remapping(tensorflow::RewriterConfig::OFF);
config->set_pin_to_host_optimization(tensorflow::RewriterConfig::OFF);
config->mutable_auto_parallel()->set_enable(false);
config->clear_optimizers();
// Create tensorflow session with customized options
std::unique_ptr<tensorflow::Session> tf_session(tensorflow::NewSession(session_opts));