Skip to content
/ NYC-Open-Dataset-Analysis Public

Identified data types for each distinct column value on 1900 data sets. For each column, summarized semantic types present in the column, using Fuzzy Logic, Levenshtein distance. Identified & derived inference the 3 most frequent 311 complaint types by borough.

1 star 0 forks Branches Tags Activity
Star
Notifications

p-disha/NYC-Open-Dataset-Analysis

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits

.gitattributes

.gitattributes

 
 

README.md

README.md

 
 

col_names_line_separated.txt

col_names_line_separated.txt

 
 

genericDataProfiling.py

genericDataProfiling.py

 
 

json_processing.py

json_processing.py

 
 

manual labelling - Sheet1.csv

manual labelling - Sheet1.csv

 
 

manual_labeling.csv

manual_labeling.csv

 
 

manual_labeling.tsv

manual_labeling.tsv

 
 

semanticProfiling.py

semanticProfiling.py

 
 

task2-manual-labels.json

task2-manual-labels.json

 
 

task2.json

task2.json

 
 

task2_manual_labeling.py

task2_manual_labeling.py

 
 

task2_scratch_code.py

task2_scratch_code.py

 
 

task3_final.py

task3_final.py

 
 

temp_data.txt

temp_data.txt

 
 

Repository files navigation

   



        


  
      

      

        

          

            

  
About


      

        Identified data types for each distinct column value on 1900 data sets. For each column, summarized semantic types present in the column, using Fuzzy Logic, Levenshtein distance. Identified & derived inference the 3 most frequent 311 complaint types by borough.
      


    
Topics

    

        

      
  visualization

      
  json

      
  big-data

      
  python3

      
  pyspark

      
  levenshtein-distance

      
  matplotlib

      
  fuzzy-logic

      
  data-profiling

      
  big-data-analytics

      
  nyc-opendata

      
  nyc-open-data

      
  311-data

  


    


    
Resources

    

      
        
        Readme
    


  



  
  

  

    
      
      Activity
  



  
Stars

  

    
      
      1
      star
  


  
Watchers

  

    
      
      1
      watching
  


  
Forks

  

    
      
      0
      forks
  


    

      
          Report repository
    




          

        


        
            

              

                

  
    Releases



    
No releases published


              

            


        
        
            

              

                

  
    Packages
      




      

        No packages published 

      




              

            


        
            

        
        
        
            

              

                
Languages