{"html":"<div class=\"article_cover_container\" style=\"background-image: url(https:\/\/www.bbntimes.com\/images\/articles\/global-economy\/Rethinking_Data_Science_and_Engineering.jpg);\"><div class=\"bg\"><\/div><h1>Rethinking Data Science and Engineering: Data Sets and Time Series<\/h1><\/div><style>#section-kmt{min-height: unset;}.desc-ath p {overflow: hidden;display: -webkit-box;-webkit-line-clamp: 4;-webkit-box-orient: vertical;text-align: justify;}<\/style> <div class=\"jm-allpage-space\"> <div id=\"jm-mainpage\"> <div id=\"jm-mainpage-in\"> <div id=\"jm-main\" class=\"scheme_2 clearfix\"> <div id=\"jm-content\"> <div class=\"jm-content-in\"> <div> <\/div> <\/div> <\/div> <\/div> <\/div> <\/div> <div id=\"jm-mainpage\"> <div id=\"jm-mainpage-in\"> <div id=\"jm-main\" class=\"scheme_2 clearfix\"> <div id=\"jm-content\"> <div class=\"jm-content-in\"> <div id=\"jm-maincontent\"> <div class=\"cont-maincontent fadeIn\"> <div class=\"item-page\"> <div class=\"article-tittle\"> <h2>Rethinking Data Science and Engineering: Data Sets and Time Series<\/h2> <\/div> <div class=\"top-infos-article\"> <div class=\"left-top\"> <span class=\"author-article\"> <a href=\"\/azamat-abdoullaev\" class=\"author_img\" lazy-style=\"background-image: url('https:\/\/www.bbntimes.com\/\/images\/authors\/Azamat.jpg');\"> <\/a> <a class=\"author_name\" href=\"\/azamat-abdoullaev\"> Azamat Abdoullaev <\/a> <\/span> <span class=\"date-publication\">26\/01\/2022<\/span> <\/div> <\/div> <div class=\"bloc-content\"> <div class=\"wv-button-placeholder\"><\/div> <div class=\"pull-left item-image\"> <img src=\"https:\/\/www.bbntimes.com\/\/images\/articles\/global-economy\/Rethinking_Data_Science_and_Engineering.jpg\" alt=\"Rethinking Data Science and Engineering: Data Sets and Time Series\" itemprop=\"image\" \/> <\/div> <p>Organisations are revamping their data science and engineering strategies to gain the necessary skills to&nbsp;deploy <a href=\"technology\/the-latest-trends-in-artificial-intelligence-ai-and-machine-learning-ml\" target=\"_blank\" rel=\"noopener\">artificial intelligence (AI) and machine learning (ML) <\/a>systems. <p>Companies are now hiring legions of data scientists and other data experts to build&nbsp;<a href=\"technology\/how-big-data-and-artificial-intelligence-can-create-new-possibilities\" target=\"_blank\" rel=\"noopener\">artificial intelligence<\/a>, <a href=\"financial\/the-growing-role-of-machine-learning-in-fraud-detection\" target=\"_blank\" rel=\"noopener\">machine learning<\/a> and <a href=\"technology\/to-leverage-deep-learning-you-must-know-this-first\" target=\"_blank\" rel=\"noopener\">deep learning<\/a> (DL) applications, trained&nbsp;analytics translators&nbsp;to connect the business and technical realms, and upskilled frontline staff to use advanced technological applications effectively. <p>One role in particular, the <a href=\"society\/everything-you-need-to-know-data-scientist-vs-machine-learning-engineer\" target=\"_blank\" rel=\"noopener\">data scientist<\/a>, has been especially difficult for leaders to fill as competition for its illusive knowledge increased.<\/p> <p><img src=\"https:\/\/www.bbntimes.com\/\/images\/Data_Science_Process.jpg\" alt=\"Data_Science_Process.jpg\" width=\"788\" height=\"603\" \/><br \/><br \/>Data science and engineering is a transdisciplinary field that uses scientific methods, processes, algorithms and systems to extract causal knowledge from structured and unstructured data, and apply causal insights from data across a broad range of application domains. <p>The combination of data science and engineering can help&nbsp;design and build data\/information processing systems in information technology, digital electronic computers, artificial intelligence, machine learning, deep learning and even data analytics systems. <p>Data science and engineering aims&nbsp;to unify computer science and information science, mathematics and statistics, data analysis and empirical science, all within data typology (ontology) frameworks, to understand and analyse real world data.&nbsp;<\/p> <p><img src=\"https:\/\/www.bbntimes.com\/\/images\/Data_Science_Evolution.png\" alt=\"Data_Science_Evolution.png\" width=\"963\" height=\"465\" \/> <h2>Data Sets in&nbsp;the Context of Data Science and Engineering<\/h2> <p><span style=\"color: #000000;\">A data set is a collection of one or more tables, schemas, points, and\/or objects that are grouped together either because they\u2019re stored in the same location or because they\u2019re related to the same subject.<\/span> <p><span style=\"color: #000000;\">Several classic data sets have been used extensively in the&nbsp;statistical&nbsp;literature:<\/span> <header> <section> <ul> <li> <p>Iris flower data set&nbsp;\u2013 Multivariate data set introduced by&nbsp;Ronald Fisher&nbsp;(1936). <\/p><\/li> <li> <p>MNIST database&nbsp;\u2013 Images of handwritten digits commonly used to test classification, clustering, and image processing algorithms <\/p><\/li> <li> <p>Categorical data analysis&nbsp;\u2013 Data sets used in the book,&nbsp;An Introduction to Categorical Data Analysis. <\/p><\/li> <li> <p>Robust statistics&nbsp;\u2013 Data sets used in&nbsp;Robust Regression and Outlier Detection&nbsp;(Rousseeuw&nbsp;and Leroy, 1986).&nbsp;Provided on-line at the University of Cologne. <\/p><\/li> <li> <p>Time series&nbsp;\u2013 Data used in Chatfield's book,&nbsp;The Analysis of Time Series, are&nbsp;provided on-line by StatLib. <\/p><\/li> <li> <p>Extreme values&nbsp;\u2013 Data used in the book,&nbsp;An Introduction to the Statistical Modeling of Extreme Values&nbsp;are&nbsp;a snapshot of the data as it was provided on-line by Stuart Coles, the book's author. <\/p><\/li> <li> <p>Bayesian Data Analysis&nbsp;\u2013 Data used in the book are&nbsp;provided on-line&nbsp;by&nbsp;Andrew Gelman, one of the book's authors. <\/p><\/li> <li> <p>The&nbsp;Bupa liver data&nbsp;\u2013 Used in several papers in the machine learning (data mining) literature. <\/p><\/li> <li> <p>Anscombe's quartet&nbsp;\u2013 Small data set illustrating the importance of graphing the data to avoid statistical fallacies <\/p><\/li> <\/ul> <\/section> <\/header> <h2>Time Series in the Context of Data Science and Engineering<\/h2> <p>A time series is a series of data points indexed (or listed or graphed or plotted via run charts (a temporal line chart)) in time order; a sequence taken at successive equally spaced points in time; a sequence of discrete-time data, eg, letters and words, seasonal precipitation, heights of ocean tides, counts of sunspots, the Dow Jones Industrial Average, etc. <p>Time series are used in any domain of applied science and engineering which involves temporal measurements, statistics, signal processing, pattern recognition, econometrics, mathematical finance, weather forecasting, earthquake prediction, electroencephalography, control engineering, astronomy, communications engineering. <p>In the context of pattern recognition and machine learning, time series analysis can be used for clustering, classification, query by content, anomaly detection as well as forecasting. <p>A time series has a natural temporal ordering applied to real-valued, continuous data, discrete numeric data, or discrete symbolic data (sequences of characters, such as letters and words) with the primary goal of time series analysis to forecast. <p>The Granger statistical causality test, which is not any causality, is widely applied in econometrics to determine whether one time series is useful in forecasting another. Rather than testing whether Y causes X, the Granger causality tests whether Y forecasts X. <p>When time series X Granger-causes time series Y, the patterns in X are approximately repeated in Y after some time lag. Thus, past values of X can be used for the prediction of future values of Y. <p>In all, the time series data is a type of panel data, the general class, a multidimensional data set. <p>A number of tools for investigating time-series data are hard to count: <ul> <li> <p>Principal component analysis (or empirical orthogonal function analysis) <\/p><\/li> <li> <p>Singular spectrum analysis <\/p><\/li> <li> <p>General State Space Models <\/p><\/li> <li> <p>Dynamic time warping <\/p><\/li> <li> <p>Cross-correlation <\/p><\/li> <li> <p>Dynamic Bayesian network <\/p><\/li> <li> <p>Time-frequency analysis techniques <\/p><\/li> <li> <p>Machine Learning <\/p><\/li> <li> <p>Artificial neural networks <\/p><\/li> <li> <p>Support vector machine <\/p><\/li> <li> <p>Fuzzy logic <\/p><\/li> <li> <p>Gaussian process <\/p><\/li> <li> <p>Hidden Markov model <\/p><\/li> <\/ul> <p>As a combination of time series analysis and cross-sectional analysis, the panel data looks as more meaningful or informative, being widely used in the social sciences with a view: <ol> <li> <p>to understand causation of events that may have occurred in the past and how they lead to outcomes observed in later waves of the data, as the effect of the passage a new law on crime statistics, or a natural disaster on births and deaths years later, or how stock prices react to merger and earnings announcements; <\/p><\/li> <li> <p>to track trends and changes over time by asking the same respondents questions in several waves carried out of time, as to measure poverty and income inequality by tracking individual households; <\/p><\/li> <li> <p>to track company profitability, risk, and to understand the effect of economic shocks or determine the factors that most affect unemployment; <\/p><\/li> <li> <p>to perform regression analysis, as to determine how many specific factors such as the price of a commodity, interest rates, particular industries, or sectors influence the price movement of an asset; <\/p><\/li> <li> <p>to event study to analyse the impact of an event, as Covid-19, on an industry, sector or the overall market, etc. <\/p><\/li> <\/ol> <p>The time series data is studied by data science and engineering&nbsp;, which is all about creating value from data; to better understand data relationships, uncover insights, predict future trends and behaviour and make intelligent decisioning. <h2>Conclusion<\/h2> <p>Organisations are finding it difficult to compete with start-ups and tech giants such as Google, Meta and Twitter to attract or retain the best practising data scientists and the newest crop of graduates.&nbsp; <p>Due to the <a href=\"companies\/insights-on-demand-post-covid-19-pandemic-moving-from-resilience-to-growth-investing-in-data-saving-on-storage\" target=\"_blank\" rel=\"noopener\">Covid-19 pandemic<\/a>,&nbsp;tech start-ups may struggle to survive, making it somewhat easier for incumbents to acquire these hard-to-get skills. <p><img src=\"https:\/\/www.bbntimes.com\/\/images\/How_AutoMl_Works.png\" alt=\"How_AutoMl_Works.png\" width=\"947\" height=\"452\" \/> <p><br \/>There are also new tools that have the potential to fill the data-science talent gap and increase the efficiency of analytics teams. Automated machine learning (ML) tools, commonly called AutoML, are designed to automate many steps in developing machine learning models.&nbsp; <p>Data science and engineering require deep thinking &amp; multi-discipline collaboration.&nbsp; <p>In order to stay competitive, organizations will be best served by not putting all their resources into the fight for sparse technical talent, but instead focusing at least part of their attention on building up their troop of <a href=\"companies\/how-google-uses-machine-learning\" target=\"_blank\" rel=\"noopener\">AutoML <\/a>practitioners, who will become a substantial proportion of the talent pool for the next decade. <\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/p><\/div> <div class=\"moduletable bbb\"> <div class=\"module-tittle\"> <h3>Share this article<\/h3> <\/div> <div class=\"module-content  bbb\"> <ul class='lg-share dynamicul'> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series&title=Rethinking Data Science and Engineering: Data Sets and Time Series\" title=\"facebook\"><i class=\"rounded fa fa-facebook\"><\/i><\/a><\/li> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/twitter.com\/share?url=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series&text=Rethinking Data Science and Engineering: Data Sets and Time Series&via=BBNTimes_en\" title=\"Twitter\"><i class=\"rounded fa fa-twitter\"><\/i><\/a><\/li> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/shareArticle?title=Rethinking Data Science and Engineering: Data Sets and Time Series&url=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series\" title=\"linkedin\"><i class=\"rounded fa fa-linkedin\"><\/i><\/a><\/li> <li> <a class=\"share-this\" href=\"mailto:?subject=I wanted you to see this site&amp;body=Check out this site https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series\" title=\"Share by Email\"> <i class=\"rounded fa fa-envelope\"><\/i> <\/a> <\/li> <\/ul> <\/div> <input type=\"hidden\" class=\"admin_path\" value=\"\/var\/www\/vhosts\/bbntimes.com\/httpdocs\" \/> <\/div> <div id=\"section-kmt\" class=\"theme-kuro\"><div id=\"kmt-form\" class=\"commentForm kmt-form \"><div class=\"formArea kmt-form-area\"><a href=\"\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series?no-scroll=true#section-kmt\" title=\"Leave or view comments\"><h3 class=\"kmt-title\">Leave or view comments <\/h3><\/a><\/div><\/div><\/div> <span id=\"komento-token\" style=\"display:none;\"><input type=\"hidden\" name=\"6acbb35a7d5ac83b72cb79080713e6ee\" value=\"1\" \/><\/span> <\/div> <\/div> <\/div> <\/div> <\/div> <div id=\"jm-right\"> <div class=\"jm-right-in\"> <div class=\"jm-module right-share\" id=\"Mod404\"> <div class=\"jm-module-in\"> <div class=\"jm-module-content clearfix\"> <div class=\"module-tittle\"> <h3>Share this article<\/h3> <\/div> <div class=\"module-content right-share\"> <ul class='lg-share dynamicul'> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series&title=Rethinking Data Science and Engineering: Data Sets and Time Series\" title=\"facebook\"><i class=\"rounded fa fa-facebook\"><\/i><\/a><\/li> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/twitter.com\/share?url=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series&text=Rethinking Data Science and Engineering: Data Sets and Time Series&via=BBNTimes_en\" title=\"Twitter\"><i class=\"rounded fa fa-twitter\"><\/i><\/a><\/li> <li><a class=\"share-this\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/shareArticle?title=Rethinking Data Science and Engineering: Data Sets and Time Series&url=https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series\" title=\"linkedin\"><i class=\"rounded fa fa-linkedin\"><\/i><\/a><\/li> <li> <a class=\"share-this\" href=\"mailto:?subject=I wanted you to see this site&amp;body=Check out this site https:\/\/www.bbntimes.com\/technology\/rethinking-data-science-and-engineering-data-sets-and-time-series\" title=\"Share by Email\"> <i class=\"rounded fa fa-envelope\"><\/i> <\/a> <\/li> <\/ul> <\/div> <input type=\"hidden\" class=\"admin_path\" value=\"\/var\/www\/vhosts\/bbntimes.com\/httpdocs\" \/> <\/div> <\/div> <\/div> <div class=\"jm-module \" id=\"Mod476\"> <div class=\"jm-module-in\"> <div class=\"jm-module-content clearfix\"> <div class=\"containt-atinf\"> <div class=\"top-infos\"> <div class=\"cols-imgath\" lazy-style=\"background-image: url('https:\/\/www.bbntimes.com\/\/images\/authors\/Azamat.jpg');\"> <\/div> <div class=\"cols-infoath\"> <h4>Azamat Abdoullaev<\/h4> <h5>Tech Expert <\/h5> <\/div> <\/div> <div class=\"desc-ath\"> <p><span style=\"color: #000000;\">Azamat Abdoullaev&nbsp;is a leading ontologist&nbsp;and&nbsp;theoretical physicist&nbsp;who introduced a universal&nbsp;world&nbsp;model&nbsp;as a&nbsp;standard ontology\/semantics&nbsp;for&nbsp;human beings&nbsp;and&nbsp;computing&nbsp;machines. He holds a Ph.D. in mathematics and theoretical physics.&nbsp;<\/span><\/p> <\/div> <div class=\"icons\"> <a class=\"user-articles\" href=\"\/azamat-abdoullaev\">&nbsp;<\/a> <a class=\"fa fa-linkedin\" href=\"https:\/\/www.linkedin.com\/in\/azamat-abdoullaev-335a0881\/\" target=\"_blank\" rel=\"noopener noreferrer\">&nbsp;<\/a> <\/div> <\/div> <\/div> <\/div> <\/div> <div class=\"jm-module  trending-right-column\" id=\"Mod506\"> <div class=\"jm-module-in\"> <div class=\"jm-module-content clearfix\"> <div class=\"bloc-tittle\"> <h3>Trending<\/h3> <hr \/> <\/div> <ul class=\"category-module trending-right-column\"> <li> <h4> <span class=\"index\">1<\/span> <a class=\"mod-articles-trending-title \" rel=\"nofollow\" href=\"\/technology\/can-job-guarantee-networking-courses-solve-the-skills-gap-crisis\">Can Job Guarantee Networking Courses Solve the Skills Gap Crisis?<\/a> <\/h4> <a href=\"\/luke-fitzpatrick\"> <span class=\"mod-articles-cat-latest-writtenby auteur\"> Luke Fitzpatrick <\/span> <\/a> <\/li> <li> <h4> <span class=\"index\">2<\/span> <a class=\"mod-articles-trending-title \" href=\"\/technology\/snowflake-unveils-arctic-a-breakthrough-enterprise-language-model\">Snowflake Inc. Unveils Arctic: A Breakthrough Enterprise Language Model<\/a> <\/h4> <a href=\"\/azamat-abdoullaev\"> <span class=\"mod-articles-cat-latest-writtenby auteur\"> Azamat Abdoullaev <\/span> <\/a> <\/li> <li> <h4> <span class=\"index\">3<\/span> <a class=\"mod-articles-trending-title \" href=\"\/technology\/how-to-win-the-next-generation-of-air-travelers-through-personalized-offers\">How to Win the Next Generation of Air Travelers Through Personalized Offers<\/a> <\/h4> <a href=\"\/naveen-joshi\"> <span class=\"mod-articles-cat-latest-writtenby auteur\"> Naveen Joshi <\/span> <\/a> <\/li> <li> <h4> <span class=\"index\">4<\/span> <a class=\"mod-articles-trending-title \" href=\"\/technology\/google-faces-setback-in-cookie-deprecation-plan-amid-ad-tech-industry-pressure\">Google Faces Setback in Cookie Deprecation Plan Amid Ad Tech Industry Pressure<\/a> <\/h4> <a href=\"\/azamat-abdoullaev\"> <span class=\"mod-articles-cat-latest-writtenby auteur\"> Azamat Abdoullaev <\/span> <\/a> <\/li> <li> <h4> <span class=\"index\">5<\/span> <a class=\"mod-articles-trending-title \" href=\"\/technology\/can-tech-meet-tradition-the-modern-couple-s-guide-to-a-digital-savvy-wedding\">Can Tech Meet Tradition? The Modern Couple's Guide to a Digital-Savvy Wedding<\/a> <\/h4> <a href=\"\/mihir-gadhvi\"> <span class=\"mod-articles-cat-latest-writtenby auteur\"> Mihir Gadhvi <\/span> <\/a> <\/li> <\/ul> <\/div> <\/div> <\/div> <div class=\"jm-module  related_articles_tag\" id=\"Mod504\"> <div class=\"jm-module-in\"> <div class=\"jm-module-content clearfix\"> <h3 class=\"mod-title\">Related articles<\/h3> <ul class=\"relateditems supertag related_articles_tag\"> <li> <div class=\"image_intro\"> <img class=\"image_intro gi\" src-lazy=\"\/images\/articles\/society\/resized\/The_Transformation_of_Medical_Alert_Systems_Through_Satellite_Technology[img-size].jpg\" alt=\"\" \/> <a class=\"article_link\" href=\"\/technology\/the-transformation-of-medical-alert-systems-through-satellite-technology\"><\/a> <a class=\"category_name\" href=\"\/technology\">Technology<\/a> <\/div> <a class=\"article_link title\" href=\"\/technology\/the-transformation-of-medical-alert-systems-through-satellite-technology\">The Transformation of Medical Alert Systems Through Satellite Technology <\/a> <div class=\"info\"> <a href=\"\/fabrice-beaux\">Fabrice Beaux<\/a> <span class=\"nb_comments\">0<\/span> <\/div> <\/li> <li> <div class=\"image_intro\"> <img class=\"image_intro gi\" src-lazy=\"\/images\/articles\/society\/resized\/Elon_Musks_xAI_Nearing_6B_Funding_X_Social_Network_Among_Shareholders[img-size].jpg\" alt=\"\" \/> <a class=\"article_link\" href=\"\/technology\/elon-musk-s-xai-nearing-6b-funding-x-social-network-among-shareholders\"><\/a> <a class=\"category_name\" href=\"\/technology\">Technology<\/a> <\/div> <a class=\"article_link title\" href=\"\/technology\/elon-musk-s-xai-nearing-6b-funding-x-social-network-among-shareholders\">Elon Musk's xAI Nearing $6B Funding, X Social Network Among Shareholders<\/a> <div class=\"info\"> <a href=\"\/fabrice-beaux\">Fabrice Beaux<\/a> <span class=\"nb_comments\">0<\/span> <\/div> <\/li> <li> <div class=\"image_intro\"> <img class=\"image_intro gi\" src-lazy=\"\/images\/articles\/society\/resized\/Snowflake_Unveils_Arctic__A_Breakthrough_Enterprise_Language_Model[img-size].jpg\" alt=\"\" \/> <a class=\"article_link\" href=\"\/technology\/snowflake-unveils-arctic-a-breakthrough-enterprise-language-model\"><\/a> <a class=\"category_name\" href=\"\/technology\">Technology<\/a> <\/div> <a class=\"article_link title\" href=\"\/technology\/snowflake-unveils-arctic-a-breakthrough-enterprise-language-model\">Snowflake Inc. Unveils Arctic: A Breakthrough Enterprise Language Model<\/a> <div class=\"info\"> <a href=\"\/azamat-abdoullaev\">Azamat Abdoullaev<\/a> <span class=\"nb_comments\">0<\/span> <\/div> <\/li> <li> <div class=\"image_intro\"> <img class=\"image_intro gi\" src-lazy=\"\/images\/articles\/global-economy\/resized\/How_to_Win_the_Next_Generation_of_Air_Travelers_Through_Personalized_Offers[img-size].jpg\" alt=\"\" \/> <a class=\"article_link\" href=\"\/technology\/how-to-win-the-next-generation-of-air-travelers-through-personalized-offers\"><\/a> <a class=\"category_name\" href=\"\/technology\">Technology<\/a> <\/div> <a class=\"article_link title\" href=\"\/technology\/how-to-win-the-next-generation-of-air-travelers-through-personalized-offers\">How to Win the Next Generation of Air Travelers Through Personalized Offers<\/a> <div class=\"info\"> <a href=\"\/naveen-joshi\">Naveen Joshi<\/a> <span class=\"nb_comments\">0<\/span> <\/div> <\/li> <\/ul> <\/div> <\/div> <\/div> <\/div> <\/div> <\/div> <\/div> <\/div> <\/div> ","nextAricle":"\/technology\/how-to-overcome-the-challenges-to-data-integration-in-iot?scroll-ajax=html"}